View Source

https://dev.fiz-karlsruhe.de/stash/projects/APD/repos/ise-apd-classification/browse

Das Projekt enthält ein Dockerfile so dass aus dem Projekt ein Docker-Container gebaut werden kann.

DI-Netz: ddb-di-vm08, ausgecheckt in /data/ddb/tools (git clone https://dev.fiz-karlsruhe.de/stash/scm/apd/ise-apd-classification.git)

Docker installiert https://docs.docker.com/engine/install/

Python installieren?

Autostart:

sudo systemctl enable docker.service
sudo systemctl enable containerd.service
sudo usermod -a -G docker admin #um den Dockerdaemon ansprechen zu dürfen

Default-Directory

Ist normalerweise /var/lib/docker. Dort gibt es aber nicht genug Plattenplatz. Darum in /usr/lib/systemd/system/docker.service in die Zeile
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
-g ... einfügen:
ExecStart=/usr/bin/dockerd -g /data/ddb/tools/docker -H fd:// --containerd=/run/containerd/containerd.sock
und service docker neu starten

Proxy:

Als root neues Verzeichnis + Datei /etc/systemd/system/docker.service.d/http-proxy.conf anlegen
Inhalt:

[Service]
Environment="HTTP_PROXY=http://proxy.fiz-karlsruhe.de:8888"
Environment="HTTPS_PROXY=http://proxy.fiz-karlsruhe.de:8888"
Environment="NO_PROXY=localhost,127.0.0.1"

Zusätzlich neues Verzeichnis + Datei <home>/.docker/config.json Inhalt:

{
 "proxies":
 {
   "default":
   {
     "httpProxy": "http://proxy.fiz-karlsruhe.de:8888",
     "httpsProxy": "http://proxy.fiz-karlsruhe.de:8888",
     "noProxy": "*.test.example.com,.example2.com,127.0.0.0/8"
   }
 }
}

Build + Run:

Also Refer to https://dev.fiz-karlsruhe.de/stash/projects/APD/repos/ise-apd-classification/browse

cd /data/ddb/tools/ise-apd-classification
docker build -t apdweimar:latest .

Run:

Adapt configuration (Endpoints etc) in /data/ddb/tools/ise-apd-classification/config.py

Load Data:

Either from API (ddbIds are contained in /data/ddb/tools/ise-apd-classification/data/onjectIds.csv) or from Files (contained in /data/ddb/tools/ise-apd-classification/data/dump)
objectIds or Files have to get delivered by Archives.

cd /data/ddb/tools/ise-apd-classification

#From API
docker run --rm -v "$(pwd)"/data:/usr/src/app/data --name APDclassifier apdweimar:latest python loadData.py

#From Files (Path to dumpfiles is defined in /data/ddb/tools/ise-apd-classification/config.py)
docker run --rm -v "$(pwd)"/data:/usr/src/app/data --name APDclassifier apdweimar:latest python loadData.py --local

#Output is in /data/ddb/tools/ise-apd-classification/data/input

Create Suggestions:

cd /data/ddb/tools/ise-apd-classification
docker run --rm -v "$(pwd)"/data:/usr/src/app/data --name APDclassifier apdweimar:latest python classification.py -m TfIdfClassifier -s data/models/tfidf.pickle data/output/predictions.csv
# Maybe use different model (-m) and load existing trained model instead of creating a new one (-l instead of -s)

Which model to use? When to save a new trained model?

Load Suggestions:

# Configure Assignment-Tool-Backend URL in /data/ddb/tools/ise-apd-classification/config.py
 docker run --rm -v "$(pwd)"/data:/usr/src/app/data --name APDclassifier apdweimar:latest python uploadPredictions.py data/output/predictions.csv

FEHLER beim Purge: 2022-04-07 09:46:30.500 ERROR [d.f.d.a.s.AssignmentToolBackendExceptionMapper] - URL: http://dev-apd.fiz-karlsruhe.de/assignment-tool-backend/keyword-relation, Method: DELETE, Message: NotAllowedException: HTTP 405 Method Not Allowed

FEHLER beim Laden: 2022-04-07 09:49:43.335 ERROR [d.f.d.a.s.AssignmentToolBackendExceptionMapper] - URL: http://dev-apd.fiz-karlsruhe.de/assignment-tool-backend/keyword-relation/, Method: POST, Message: NotFoundException: c-012