You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 21 Next »

https://dev.fiz-karlsruhe.de/stash/projects/APD/repos/ise-apd-classification/browse

Das Projekt enthält ein Dockerfile so dass aus dem Projekt ein Docker-Container gebaut werden kann.

DI-Netz: ddb-di-vm08, ausgecheckt in /data/ddb/tools (git clone https://dev.fiz-karlsruhe.de/stash/scm/apd/ise-apd-classification.git)

Docker installiert https://docs.docker.com/engine/install/

Python installieren?

Autostart: 

sudo systemctl enable docker.service
sudo systemctl enable containerd.service
sudo usermod -a -G docker admin #um den Dockerdaemon ansprechen zu dürfen

Default-Directory

Ist normalerweise /var/lib/docker. Dort gibt es aber nicht genug Plattenplatz. Darum in /usr/lib/systemd/system/docker.service in die Zeile
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
-g ... einfügen:
ExecStart=/usr/bin/dockerd -g /data/ddb/tools/docker -H fd:// --containerd=/run/containerd/containerd.sock
und service docker neu starten

Proxy:

Als root neues Verzeichnis + Datei /etc/systemd/system/docker.service.d/http-proxy.conf anlegen
Inhalt:


[Service]
Environment="HTTP_PROXY=http://proxy.fiz-karlsruhe.de:8888"
Environment="HTTPS_PROXY=http://proxy.fiz-karlsruhe.de:8888"
Environment="NO_PROXY=localhost,127.0.0.1"


Zusätzlich neues Verzeichnis + Datei <home>/.docker/config.json
Inhalt:

{
 "proxies":
 {
   "default":
   {
     "httpProxy": "http://proxy.fiz-karlsruhe.de:8888",
     "httpsProxy": "http://proxy.fiz-karlsruhe.de:8888",
     "noProxy": "*.test.example.com,.example2.com,127.0.0.0/8"
   }
 }
}

Build + Run:

Also Refer to https://dev.fiz-karlsruhe.de/stash/projects/APD/repos/ise-apd-classification/browse

Build
cd /data/ddb/tools/ise-apd-classification
docker build -t apdweimar:latest .

Run:

Adapt configuration (Endpoints etc) in /data/ddb/tools/ise-apd-classification/config.py

Load Data:

  • Either from API (ddbIds are contained in /data/ddb/tools/ise-apd-classification/data/onjectIds.csv) or from Files (contained in /data/ddb/tools/ise-apd-classification/data/dump)
  • objectIds or Files have to get delivered by Archives.
  • Load Data
    cd /data/ddb/tools/ise-apd-classification

#From API
docker run --rm -v "$(pwd)"/data:/usr/src/app/data --name APDclassifier apdweimar:latest python loadData.py

#From Files (Path to dumpfiles is defined in /data/ddb/tools/ise-apd-classification/config.py)
docker run --rm -v "$(pwd)"/data:/usr/src/app/data --name APDclassifier apdweimar:latest python loadData.py --local

#Output is in /data/ddb/tools/ise-apd-classification/data/input

Create Suggestions:

  • Create Suggestions
    cd /data/ddb/tools/ise-apd-classification
docker run --rm -v "$(pwd)"/data:/usr/src/app/data --name APDclassifier apdweimar:latest python classification.py -m TfIdfClassifier -s data/models/tfidf.pickle data/output/predictions.csv
# Maybe use different model (-m) and load existing trained model instead of creating a new one (-l instead of -s)

    Which model to use? When to save a new trained model?

Load Suggestions:

  • Load Suggestions
    # Configure Assignment-Tool-Backend URL in /data/ddb/tools/ise-apd-classification/config.py
 docker run --rm -v "$(pwd)"/data:/usr/src/app/data --name APDclassifier apdweimar:latest python uploadPredictions.py data/output/predictions.csv

    FEHLER beim Purge: 2022-04-07 09:46:30.500 ERROR [d.f.d.a.s.AssignmentToolBackendExceptionMapper] - URL: http://dev-apd.fiz-karlsruhe.de/assignment-tool-backend/keyword-relation, Method: DELETE, Message: NotAllowedException: HTTP 405 Method Not Allowed

  • No labels