Difference between revisions of "Dev:AI/CVAT"
(→Log) |
(→Install) |
||
(2 intermediate revisions by the same user not shown) | |||
Line 288: | Line 288: | ||
:/mnt/cvat/baldr/concopy | :/mnt/cvat/baldr/concopy | ||
+ | |||
+ | === Local Browser === | ||
+ | |||
+ | sudo apt install chromium-browser | ||
+ | |||
+ | |||
+ | ssh -Y -l regogr -p 23122 10.112.46.3 | ||
+ | chromium-browser | ||
+ | |||
+ | === Good to have === | ||
+ | sudo apt install nmap | ||
+ | nmap localhost | ||
+ | nmap 192.168.122.31 | ||
+ | nmap 192.168.122.31 -p8000-9000 | ||
+ | |||
+ | wget 192.168.122.31:8080 | ||
+ | docker-compose down | ||
+ | nano docker-compose.yml | ||
+ | docker-compose up -d | ||
=== ConCopy === | === ConCopy === | ||
Line 319: | Line 338: | ||
cvat_logs:/mnt/cvat/baldr/django/logs<br> | cvat_logs:/mnt/cvat/baldr/django/logs<br> | ||
cvat_models:/mnt/cvat/baldr/django/models<br> | cvat_models:/mnt/cvat/baldr/django/models<br> | ||
+ | |||
+ | |||
+ | Fredrik funkar | ||
+ | <source> | ||
+ | # | ||
+ | # Copyright (C) 2018 Intel Corporation | ||
+ | # | ||
+ | # SPDX-License-Identifier: MIT | ||
+ | # | ||
+ | version: "2.3" | ||
+ | |||
+ | services: | ||
+ | cvat_db: | ||
+ | container_name: cvat_db | ||
+ | image: postgres:10.3-alpine | ||
+ | networks: | ||
+ | default: | ||
+ | aliases: | ||
+ | - db | ||
+ | restart: always | ||
+ | environment: | ||
+ | POSTGRES_USER: root | ||
+ | POSTGRES_DB: cvat | ||
+ | volumes: | ||
+ | - cvat_db:/var/lib/postgresql/data | ||
+ | |||
+ | cvat_redis: | ||
+ | container_name: cvat_redis | ||
+ | image: redis:4.0.5-alpine | ||
+ | networks: | ||
+ | default: | ||
+ | aliases: | ||
+ | - redis | ||
+ | restart: always | ||
+ | |||
+ | cvat: | ||
+ | container_name: cvat | ||
+ | image: cvat | ||
+ | restart: always | ||
+ | depends_on: | ||
+ | - cvat_redis | ||
+ | - cvat_db | ||
+ | ports: | ||
+ | - "8080:8080" | ||
+ | build: | ||
+ | context: . | ||
+ | args: | ||
+ | http_proxy: | ||
+ | https_proxy: | ||
+ | no_proxy: | ||
+ | socks_proxy: | ||
+ | TF_ANNOTATION: "no" | ||
+ | AUTO_SEGMENTATION: "no" | ||
+ | USER: "django" | ||
+ | DJANGO_CONFIGURATION: "production" | ||
+ | WITH_TESTS: "no" | ||
+ | TZ: "Etc/UTC" | ||
+ | OPENVINO_TOOLKIT: "no" | ||
+ | environment: | ||
+ | DJANGO_MODWSGI_EXTRA_ARGS: "" | ||
+ | UI_PORT: 7080 | ||
+ | ALLOWED_HOSTS: "*" | ||
+ | |||
+ | |||
+ | volumes: | ||
+ | - cvat_data:/home/django/data | ||
+ | - cvat_keys:/home/django/keys | ||
+ | - cvat_logs:/home/django/logs | ||
+ | - cvat_models:/home/django/models | ||
+ | |||
+ | cvat_ui: | ||
+ | container_name: cvat_ui | ||
+ | image: nginx | ||
+ | restart: always | ||
+ | build: | ||
+ | context: . | ||
+ | args: | ||
+ | http_proxy: | ||
+ | https_proxy: | ||
+ | no_proxy: | ||
+ | socks_proxy: | ||
+ | dockerfile: Dockerfile.ui | ||
+ | environment: | ||
+ | REACT_APP_API_PROTOCOL: http | ||
+ | REACT_APP_API_HOST: localhost | ||
+ | REACT_APP_API_PORT: 8080 | ||
+ | networks: | ||
+ | default: | ||
+ | aliases: | ||
+ | - ui | ||
+ | depends_on: | ||
+ | - cvat | ||
+ | ports: | ||
+ | - "7080:80" | ||
+ | |||
+ | volumes: | ||
+ | cvat_db: | ||
+ | cvat_data: | ||
+ | cvat_keys: | ||
+ | cvat_logs: | ||
+ | cvat_models: | ||
+ | |||
+ | </source> | ||
+ | |||
+ | http://10.112.46.3:23188/admin/ | ||
+ | |||
<source> | <source> |
Revision as of 10:17, 3 December 2019
Contents
Links
https://github.com/opencv/cvat
https://c.onepanel.io/onepanel-demo/profile
Install
- Nvidia-Docker
Tutorials
- YouTube - Nikita Manovich
- Annotation Adventures: A Concept Tutorial in Adding Labels to Image Datasets
- https://www.linkedin.com/pulse/annotation-adventures-concept-tutorial-adding-labels-image-kacoroski/
OpenCV
- Object Detection Using OpenCV
Data managment
- big number of images (performance)
- Not compressing images
- Image storage in server
Install
https://github.com/opencv/cvat/blob/develop/cvat/apps/documentation/installation.md
- Install Docker
- Update pip
sudo apt-get install python3-setuptools
sudo apt-get install -y python3-pip
sudo pip3 install --upgrade pip
sudo pip install --upgrade pip
- Clone CVAT source code from the GitHub repository.
cd /home/regogr
- If git not installed:
sudo apt-get install -y git
git clone https://github.com/opencv/cvat
- Edit yml file - set volumes
- If not use default volumes
- Edit docker-compose.yml file see Install Balder
- Build docker images by default. It will take some time to download public docker image ubuntu
- 16.04 and install all necessary ubuntu packages to run CVAT server.
cd cvat
sudo docker-compose build
- Run docker containers. It will take some time to download public docker images like postgres
- 10.3-alpine, redis:4.0.5-alpine and create containers.
sudo docker-compose up -d
- You can register a user but by default it will not have rights even to view list of tasks. Thus you should create a superuser. A superuser can use an admin panel to assign correct groups to the user. Please use the command below
sudo docker exec -it cvat bash -ic 'python3 ~/manage.py createsuperuser'
- Superuser: django
- Email: rego.granlund@ri.se
- Password django
- Choose login and password for your admin account. For more information please read Django documentation.
- Google Chrome is the only browser which is supported by CVAT. You need to install it as well. Type commands below in a terminal window
wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add -
sudo sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list'
sudo apt-get update
sudo apt-get install -y google-chrome-stable
- Open the installed Google Chrome browser and go to http://localhost:8080. Type your login/password for the superuser on the login page and press the Login button. Now you should be able to create a new annotation task. Please read the CVAT user's guide for more details.
Uninstall
Delete all images
Stop and remove all processes/containers
- Stop process
sudo docker stop ContainerID
- Delete process container
sudo docker rm ContainerID
Deleta all cvat images befor reinstall
- Funkad
sudo docker rmi -f cvat
- Funkade inte.
sudo docker rmi -f nginx
sudo docker rmi -f ubuntu
sudo docker rmi -f postgres
sudo docker rmi -f redis
- Deletede med id nummer
sudo docker rmi -f
Deleta install folder
sudo rm -rfv /home/regogr/cvat
Delete data folders
sudo rm -rfv /var/lib/docker/volumes/cavt_cvat_data
sudo rm -rfv /var/lib/docker/volumes/cavt_cvat_db
sudo rm -rfv /var/lib/docker/volumes/cavt_cvat_keys
sudo rm -rfv /var/lib/docker/volumes/cavt_cvat_logs
sudo rm -rfv /var/lib/docker/volumes/cavt_cvat_models
No Docker
https://github.com/opencv/cvat/blob/master/CONTRIBUTING.md
Run
User guid [https://github.com/opencv/cvat/blob/develop/cvat/apps/documentation/user_guide.md CVAT
- Start CVAT
- In Google Chrome browser, go to http://localhost:8080.
- Superuser: django
- Password django
- Data disk on 10.112.46.3
- obdb on optimus.local fielddata/20190604152150/04/DCIM/
smbclient -L 10.112.46.3/obdb -U regogr
sudo mkdir /mnt/obdb
sudo mount -t cifs -o user=regogr //10.112.46.3/obdb /mnt/obdb
sudo mkdir /mnt/cvat
sudo mount -t cifs -o user=regogr //10.112.46.3/cvat /mnt/cvat
Admin
- Django administration
- Add User
- http://localhost:8080/admin/
- Press ACCOUNTS
- Press Email adresses - add
- Press Magnifier (Förstoringsglaset)
- Press Add user
- Username: rego
- Password:
- Set User Rights
- Did work after some attempts
- Logout
- Login
- http://localhost:8080/admin/
- In action list select user
- name: Rego
- name: Granlund
- mail: rego.granlund@ri.se
- Active: Yes
- Staff: Yes
- Superuser: Yes
- Groups: Add all
- Press Save
Volumes
The volumes containing the data are located at.
/var/lib/docker/volumes
sudo chmod 777 /var/lib/docker
sudo chmod -R 777 /var/lib/docker/volumes
Log
A log is saved at.
Example
/var/lib/docker/volumes/cvat_cvat_data/_data/2
cd /var/lib/docker/volumes/cvat_cvat_data/_data
Docker files
/home/rego/cvat
???
/home/rego/cvat/docker-compose.yml
Create New Task
Fields
- Name:
- Labels:
- Example
plants @select=type:undefined,Charlock,CreepingThistle,WildTurnip,MugWort,Dandelion,UnknowWeed,BlackBindweed,CurledDock,ScentlessMayweed,Fat-hen,Cleavers,CommonFumitory,Rape,Colt's-foot,HairyTare ~radio=quality:good,bad ~checkbox=dead:false
- Bug Tracker:
- Data set Repositury:
- Use LFS:
- If the annotation file is large, you can create a repository with LFS support.
- LFS is a feature of git (https://git-lfs.github.com/).
- If you enable it CVAT will store your annotations as LFS files in provided git repository.
- Otherwise it will store them as usual text files.
- Source:
- Z Order:
- Over Lap size:
- Segment size:
- Image Quality:
- Start Frame:
- Stop Frame:
- Frame Filter:
- Select Files:
Local Test
- In Google Chrome browser, go to http://localhost:8080.
Use data: .../obdb/work/Amanda/Annotations/Klar 20_20190604152150_05 (2).xml ...
- Name: RegoTest1
- Labels:
plants @select=type:undefined,Charlock,CreepingThistle,WildTurnip,MugWort,Dandelion,UnknowWeed,BlackBindweed,CurledDock,ScentlessMayweed,Fat-hen,Cleavers,CommonFumitory,Rape,Colt's-foot,HairyTare ~radio=quality:good,bad ~checkbox=dead:false
- Bug Tracker:
- Data set Repositury:
- Use LFS:
- Source:
- Z Order:
- Over Lap size:
- Segment size:
- Image Quality:
- Start Frame:
- Stop Frame:
- Frame Filter:
- Select Files:Selected 10 files
Did generate: http://localhost:8080/?id=1
Cvat Server
Connect
- ssh
ssh -l regogr -p 23122 10.112.46.3
- Shared folders
cd /mnt
cd /mnt/cvat
Test install 1 - Baldr
Install Folders
- Data root
- /mnt/cvat/baldr/
- Configuration copy
- /mnt/cvat/baldr/concopy
Local Browser
sudo apt install chromium-browser
ssh -Y -l regogr -p 23122 10.112.46.3
chromium-browser
Good to have
sudo apt install nmap nmap localhost nmap 192.168.122.31 nmap 192.168.122.31 -p8000-9000
wget 192.168.122.31:8080 docker-compose down nano docker-compose.yml docker-compose up -d
ConCopy
- docker-compose.yml
sudo cp /home/regogr/cvat/docker-compose.yml /mnt/cvat/baldr/concopy
sudo mv /mnt/cvat/baldr/concopy/docker-compose.yml /mnt/cvat/baldr/concopy/docker-compose-orginal.yml
cd /mnt/cvat/baldr/concopy
sudo pico /home/regogr/cvat/docker-compose.yml
Install
- Clone CVAT source code from the GitHub repository.
- Install git if needed:
sudo apt-get install -y git
git clone https://github.com/opencv/cvat
Edit docker-compose.yml
- Backup
sudo cp /home/regogr/cvat/docker-compose.yml /mnt/cvat/baldr/concopy
sudo mv /mnt/cvat/baldr/concopy/docker-compose.yml /mnt/cvat/baldr/concopy/docker-compose-orginal.yml
- Edit
sudo pico /home/regogr/cvat/docker-compose.yml
sudo cp /home/regogr/cvat/docker-compose.yml /mnt/cvat/baldr/concopy
cvat_db:/mnt/cvat/baldr/postgresql/data
cvat_data:/mnt/cvat/baldr/django/data
cvat_keys:/mnt/cvat/baldr/django/keys
cvat_logs:/mnt/cvat/baldr/django/logs
cvat_models:/mnt/cvat/baldr/django/models
Fredrik funkar
#
# Copyright (C) 2018 Intel Corporation
#
# SPDX-License-Identifier: MIT
#
version: "2.3"
services:
cvat_db:
container_name: cvat_db
image: postgres:10.3-alpine
networks:
default:
aliases:
- db
restart: always
environment:
POSTGRES_USER: root
POSTGRES_DB: cvat
volumes:
- cvat_db:/var/lib/postgresql/data
cvat_redis:
container_name: cvat_redis
image: redis:4.0.5-alpine
networks:
default:
aliases:
- redis
restart: always
cvat:
container_name: cvat
image: cvat
restart: always
depends_on:
- cvat_redis
- cvat_db
ports:
- "8080:8080"
build:
context: .
args:
http_proxy:
https_proxy:
no_proxy:
socks_proxy:
TF_ANNOTATION: "no"
AUTO_SEGMENTATION: "no"
USER: "django"
DJANGO_CONFIGURATION: "production"
WITH_TESTS: "no"
TZ: "Etc/UTC"
OPENVINO_TOOLKIT: "no"
environment:
DJANGO_MODWSGI_EXTRA_ARGS: ""
UI_PORT: 7080
ALLOWED_HOSTS: "*"
volumes:
- cvat_data:/home/django/data
- cvat_keys:/home/django/keys
- cvat_logs:/home/django/logs
- cvat_models:/home/django/models
cvat_ui:
container_name: cvat_ui
image: nginx
restart: always
build:
context: .
args:
http_proxy:
https_proxy:
no_proxy:
socks_proxy:
dockerfile: Dockerfile.ui
environment:
REACT_APP_API_PROTOCOL: http
REACT_APP_API_HOST: localhost
REACT_APP_API_PORT: 8080
networks:
default:
aliases:
- ui
depends_on:
- cvat
ports:
- "7080:80"
volumes:
cvat_db:
cvat_data:
cvat_keys:
cvat_logs:
cvat_models:
http://10.112.46.3:23188/admin/
#
# Copyright (C) 2018 Intel Corporation
#
# SPDX-License-Identifier: MIT
#
version: "2.3"
services:
cvat_db:
container_name: cvat_db
image: postgres:10.3-alpine
networks:
default:
aliases:
- db
restart: always
environment:
POSTGRES_USER: root
POSTGRES_DB: cvat
volumes:
- cvat_db:/mnt/cvat/baldr/postgresql/data
cvat_redis:
container_name: cvat_redis
image: redis:4.0.5-alpine
networks:
default:
aliases:
- redis
restart: always
cvat:
container_name: cvat
image: cvat
restart: always
depends_on:
- cvat_redis
- cvat_db
ports:
- "8080:8080"
build:
context: .
args:
http_proxy:
https_proxy:
no_proxy:
socks_proxy:
TF_ANNOTATION: "no"
AUTO_SEGMENTATION: "no"
USER: "django"
DJANGO_CONFIGURATION: "production"
WITH_TESTS: "no"
TZ: "Etc/UTC"
OPENVINO_TOOLKIT: "no"
environment:
DJANGO_MODWSGI_EXTRA_ARGS: ""
UI_PORT: 7080
volumes:
- cvat_data:/mnt/cvat/baldr/django/data
- cvat_keys:/mnt/cvat/baldr/django/keys
- cvat_logs:/mnt/cvat/baldr/django/logs
- cvat_models:/mnt/cvat/baldr/django/models
cvat_ui:
container_name: cvat_ui
image: nginx
restart: always
build:
context: .
args:
http_proxy:
https_proxy:
no_proxy:
socks_proxy:
dockerfile: Dockerfile.ui
environment:
REACT_APP_API_PROTOCOL: http
REACT_APP_API_HOST: localhost
REACT_APP_API_PORT: 8080
networks:
default:
aliases:
- ui
depends_on:
- cvat
ports:
- "7080:80"
volumes:
cvat_db:
cvat_data:
cvat_keys:
cvat_logs:
cvat_models:
Run
Error
- Error when running
sudo docker-compose build
- Command step
Processing triggers for sgml-base (1.26+nmu4ubuntu1) ... Processing triggers for dbus (1.10.6-1ubuntu3.4) ...
- Error
gpg: keyring `/tmp/tmpv4i455ob/secring.gpg' created gpg: keyring `/tmp/tmpv4i455ob/pubring.gpg' created gpg: requesting key ED8E640A from hkp server keyserver.ubuntu.com gpg: /tmp/tmpv4i455ob/trustdb.gpg: trustdb created gpg: key ED8E640A: public key "Launchpad PPA for Doug McMahon" imported gpg: no ultimately trusted keys found gpg: Total number processed: 1 gpg: imported: 1 (RSA: 1) OK gpg: keyring `/tmp/tmpy0p4jqzy/secring.gpg' created gpg: keyring `/tmp/tmpy0p4jqzy/pubring.gpg' created gpg: requesting key ED8E640A from hkp server keyserver.ubuntu.com gpg: /tmp/tmpy0p4jqzy/trustdb.gpg: trustdb created gpg: key ED8E640A: public key "Launchpad PPA for Doug McMahon" imported gpg: no ultimately trusted keys found gpg: Total number processed: 1 gpg: imported: 1 (RSA: 1) OK
- Error
You are using pip version 8.1.1, however version 19.3.1 is available. You should consider upgrading via the 'pip install --upgrade pip' command. Current default time zone: 'Etc/UTC' Local time is now: Tue Nov 26 12:00:58 UTC 2019. Universal Time is now: Tue Nov 26 12:00:58 UTC 2019.
- Error
Step 13/21 : RUN npm install ---> Running in ca75760cb59b npm WARN deprecated fsevents@1.2.9: One of your dependencies needs to upgrade to fsevents v2: 1) Proper nodejs v10+ support 2) No more fetching binaries from AWS, smaller package size npm WARN deprecated left-pad@1.3.0: use String.prototype.padStart() npm WARN deprecated circular-json@0.3.3: CircularJSON is in maintenance only, flatted is its successor. > core-js@3.4.2 postinstall /tmp/cvat-core/node_modules/core-js > node -e "try{require('./postinstall')}catch(e){}" Thank you for using core-js ( https://github.com/zloirock/core-js ) for polyfilling JavaScript standard library! The project needs your help! Please consider supporting of core-js on Open Collective or Patreon: > https://opencollective.com/core-js > https://www.patreon.com/zloirock Also, the author of core-js ( https://github.com/zloirock ) is looking for a good job -) npm notice created a lockfile as package-lock.json. You should commit this file. npm WARN acorn-jsx@5.1.0 requires a peer of acorn@^6.0.0 || ^7.0.0 but none is installed. You must install peer dependencies yourself. npm WARN cvat-core.js@0.1.0 No repository field. npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.9 (node_modules/fsevents): npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.9: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
- Error
Step 14/21 : WORKDIR /tmp/cvat-ui/ ---> Running in b42babfe132c Removing intermediate container b42babfe132c ---> 4d181675b73f Step 15/21 : RUN npm install ---> Running in bbb23d04b751 > core-js@2.6.10 postinstall /tmp/cvat-ui/node_modules/core-js > node postinstall || echo "ignore" Thank you for using core-js ( https://github.com/zloirock/core-js ) for polyfilling JavaScript standard library! The project needs your help! Please consider supporting of core-js on Open Collective or Patreon: > https://opencollective.com/core-js > https://www.patreon.com/zloirock Also, the author of core-js ( https://github.com/zloirock ) is looking for a good job -) > nodemon@1.19.4 postinstall /tmp/cvat-ui/node_modules/nodemon > node bin/postinstall || exit 0 Love nodemon? You can now support the project via the open collective: > https://opencollective.com/nodemon/donate npm WARN cvat-ui@0.1.0 No repository field. npm WARN cvat-ui@0.1.0 scripts['server'] should probably be scripts['start']. npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.9 (node_modules/fsevents): npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.9: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
Not Working
- No home directory, To create home folder I did run: sudo mkhomedir_helper regogr
- Command history does not work when connecting with ssh
- scp does not work, did try running on my computer, not from 10.112.46.3
sudo scp -p 23122 regogr@10.112.46.3:/home/regogr/cvat/docker-compose.yml /home/rego
sudo scp -p 23122 regogr@10.112.46.3:docker-compose.yml /home/rego
- sshfs does not work
psftp regogr@10.112.46.3:23122
- tab completion does not work
- Connect to //10.112.46.3/obdb /cvat
- Funkar
sudo mkdir /mnt/obdb
sudo mount -t cifs -o user=regogr //10.112.46.3/obdb /mnt/obdb
- Funkar inte
sudo mkdir /mnt/cvat
sudo mount -t cifs -o user=regogr //10.112.46.3/cvat /mnt/cvat
Work Files
- smb://optimus.local/obdb/fielddata/20190604152150/05/DCIM/100GOPRO/iframes
- smb://optimus.local/obdb/work/Amanda/Annotations