preamble
Data science development environments can be a headache to configure, running into inconsistent package versions, unfamiliar error messages, and long compile times. It's easy to hang your head and makes this first step into data science very difficult. And it's a completely uncommon barrier to entry.
Fortunately, technologies have emerged in the last few years that can solve this problem by building isolated environments. In this article, we're going to introduce a technology called Docker, which allows developers to easily and quickly build data science development environments and supports data exploration using tools such as Jupyter notebooks.
summary
Docker was originally a hobby project within dotCloud.
Docker is based on the Go language
The goal of the Docker project is to enable lightweight OS virtualization solutions
Docker is based on technologies such as Linux containers (LXC)
Docker containers can be started in seconds, which is much faster than the traditional virtual machine approach
Docker is very efficient in utilizing system resources, and thousands of Docker containers can be running on a single host at the same time
The image below compares the differences between Docker and traditional virtualization approaches, and shows that containers are virtualized at the operating system level, directly reusing the local host's operating system, whereas the traditional approach is implemented at the hardware level.
Containers basically do not consume additional system resources except for running the applications in them, making the performance of the applications very high while the system overhead is as small as possible. The traditional virtual machine approach to run 10 different applications will have to start 10 virtual machines, while Docker only needs to start 10 isolated applications.
The main advantages are:
Faster delivery and deployment - containers become the smallest unit
More Efficient Virtualization - Kernel-Level Virtualization
Easier migration and expansion
Simpler management
basic concept
There are three main ones:
Mirror (Image)
A read-only template that images can use to create Docker containers
Existing mirrors can simply be created or updated, or they can be downloaded and used directly from other people's
Container
Containers are running instances created from images, creating a writable layer as the top layer at startup (since images are read-only)
It can be started, started, stopped, and deleted. Each container is isolated from each other, a guaranteed secure platform
Think of a container as a simple version of the Linux environment (including root access, process space, user space, network space, etc.) and the applications that run in it
Repository
A centralized location for mirrored files
The largest public repository is Docker Hub
Domestic public repositories include Docker Pool, etc.
Once a user has created their image, they can use the push command to upload it to a public or private repository, so that the next time they want to use it on another machine, they just need to pull it down from the repository.
The concept of a Docker repository is similar to that of Git, and a registered server can be thought of as a hosting service like GitHub.
mounting
The official website offersMac, Linux cap (a poem)Windows The installation tutorials are for the Linux version. Since I'm going to use a virtual machine for my installation tests, I'm going to go with the Linux installation tutorial here, but the rest should be pretty much the same. My version of ubuntu is 14.04 LTS, trusty.
Docker can currently only be installed on 64-bit platforms and requires a kernel version of no less than 3.10. In fact, the newer the kernel, the better, and a kernel version that is too low tends to cause unstable functionality. You can check the kernel version with the following commands (both ways, both are fine):
parallels@ubuntu:~$ uname -a Linux ubuntu 3.13.0-83-generic #127-Ubuntu SMP Fri Mar 11 00:25:37 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
parallels@ubuntu:~$ cat /proc/version Linux version 3.13.0-83-generic (buildd@lgw01-55) (gcc version 4.8.2 (Ubuntu 4.8.2-19ubuntu1) ) #127-Ubuntu SMP Fri Mar 11 00:25:37 UTC 2016
Here we follow the tutorial on the official website without using theapt-get Instead, they usecurl to perform the installation.
1. If curl is not installed, usesudo apt-get update; sudo apt-get install curl to perform the installation
2. Download the latest Docker packagecurl -fsSL / | sh
If you want to start withnon-root user's role to use it, use thesudo usermod -aG docker parallels(parallels is the username), note that you will need to log out and log back in to apply the changes.
3、Verify the installation
It needs to be enabled firstdocker sudo service docker start
Then you can use thedocker version to view the version
final usedocker run hello-world to test
version information
parallels@ubuntu:~$ docker version Client: Version: 1.10.3 API version: 1.22 Go version: go1.5.3 Git commit: 20f81dd Built: Thu Mar 10 15:54:52 2016 OS/Arch: linux/amd64 Server: Version: 1.10.3 API version: 1.22 Go version: go1.5.3 Git commit: 20f81dd Built: Thu Mar 10 15:54:52 2016 OS/Arch: linux/amd64
Results of running hello world successfully
parallels@ubuntu:~$ docker run hello-world Hello from Docker. This message shows that your installation appears to be working correctly. To generate this message, Docker took the following steps: 1. The Docker client contacted the Docker daemon. 2. The Docker daemon pulled the "hello-world" image from the Docker Hub. 3. The Docker daemon created a new container from that image which runs the executable that produces the output you are currently reading. 4. The Docker daemon streamed that output to the Docker client, which sent it to your terminal. To try something more ambitious, you can run an Ubuntu container with: $ docker run -it ubuntu bash Share images, automate workflows, and more with a free Docker Hub account: For more examples and ideas, visit: /userguide/
The part after this is the curriculum requirements.
We need to installdocker machine respond in singingvirtual boxThe specific steps are:
# Getting access $ sudo su # Download the Docker Machine binaries $ curl -L /docker/machine/releases/download/v0.6.0/docker-machine-`uname -s`-`uname -m` > /usr/local/bin/docker-machine && chmod +x /usr/local/bin/docker-machine # Verify the installation $ docker-machine version docker-machine version 0.6.0, build e27fb87
Add Source
Add this line to the /etc/apt/ Papersdeb /virtualbox/debian trusty contrib
wget -q /download/oracle_vbox.asc -O- | sudo apt-key add -
Update and installsudo apt-get update; sudo apt-get install virtualbox-5.0
configure
Then there's some configuration, starting with resetting the default virtual machine
Remove the original default (if any)docker-machine rm default
Creating a new default virtual machinedocker-machine create --driver virtualbox default(This step may take a bit of waiting, I'm using Parallel Desktop 11 and also need to turn on CPU virtualization in the VM settings, mainly to support vt-x)
Then if you select Virtual Box directly from the menu, you will find nothing, the trick is to type virtualbox in the command line just now, and then you can see the following interface:
In Settings - Network, select Port Forwarding, and then add the record as shown here
Here, Host Port: 9234 is noted as A and Guest Port: 9001 is noted as B. Then you can start the image provided by the teacher in docker:
The command isdocker run -it -p 9001:9000 cmusvsc/apachecmda:1.1(It takes a while to download and then a while to decompress.) Here, 9001 is denoted as C and 9000 as D.
Then it will connect directly to the virtual machine in docker, and as you can see in the image afterward the command line front end has changed. Then we enter the command to turn on the front end
cd /home/Spring2016/ApacheCMDA-Frontend ./activator run # represent 9000 Port Enable
Don't close the current terminal, open a new one. Open a new terminal with
docker exec -it `docker ps -q` /bin/bash
Enter the docker being executed
After waiting for some time, continue with the following command
# Starting MySQL Server service mysql start cd /home/Spring2016/ApacheCMDA-Backend ./activator "run 9034"
And then you can see
Then wait a while (the first run needs to be compiled) and you will see the main page:
You can also try directlocalhost:9001
Remember the port forwarding from earlier? Here's an explanation:
A, or 9234, is the port number for user access
B, which is 9001, we went from 9234 to 9001 via port forwarding from the virtual machine
C, which is 9001 (docker run -it -p 9001:9000 cmusvsc/apachecmda:1.1 in this command), is the port on which the virtual machine continues to forward
D, or 9000 (docker run -it -p 9001:9000 cmusvsc/apachecmda:1.1 in this command), is the port on which requests are received in the Docker container.
That is, B and C must be the same, and A and D can be set in the command. As for why the backend needs to run on port 9034, it's because the frontend and backend communicate through this port (which is supposed to be written dead in the code)
To mount a host data volume into a container, you can perform folder mapping with the following command
docker run -it -p 9999:9999 -p 9001:9000 -v ~/localFolder:/sharedFolder cmusvsc/apachecmda:1.1
Copying files from the container to the host
# List containers docker ps # Copy, copy I don't know why this won't work docker cp <containerId>:/file/path/within/container /host/path/target # mount mapping docker run -it -v /home/parallels/Documents/code:/home/code cmusvsc/apachecmda:1.1 # Copy cp -r Spring2016/* ./code/
summarize
The above is the whole content of this article, I hope the content of this article can help you, if there are questions you can leave a message to exchange.