Tech Corner: Docker

Showing posts with label Docker. Show all posts

Thursday, June 23, 2022

How to access Postgres Database host in the Docker?

it is very convenient to spin up a Postgres Database in the docker by composing a docker compose yaml file.

the following code will help us setup a Postgres Database in the Docker

version: '3.8'
services:
  twitter_postgres_db:
    image: debezium/postgres:13-alpine
    environment:
      - POSTGRES_DATABASE=twitter_db
      - POSTGRES_USER=twitter_dev
      - POSTGRES_PASSWORD=twitter_dev
    ports:
      - 6543:5432
    volumes:
      - twitter_db_data:/data/db
volumes:
  twitter_db_data:

I should be able to setup another container for the PgAdmin in the docker. Since I already install a Postgres PgAdmin instance in my desktop. I just want to access the Postgres Database from my local host.

Here is the step that we should follow to access the Postgres from local PgAdmin

1. Register the localhost Server

2. Fill the following information in the Register Server Window

Name: LocalHost
Host Name /Address: localhost
Port: 6543 (copied from the docker compose file)
Maintenance Database: twitter_db (copied from the docker compose file)
Username: twitter_dev (copied from the docker compose file)
Password: twitter_dev (copied from the docker compose file)

3. After I click on the save button, then you should be able to access the Twttier_db

Friday, September 10, 2021

how to fix "Unable to connect to the server: dial tcp: lookup myaksclust-myresourcegroup-3f2c36-40a876eb.hcp.eastus.azmk8s.io: no such host" error?

when I try to check the version of Kubectl with kubectl version that is installed in my local desktop. i got the following error

"Unable to connect to the server: dial tcp: lookup myaksclust-myresourcegroup-3f2c36-40a876eb.hcp.eastus.azmk8s.io: no such host"

I check the kubectl configuration file with Kubectl config view

it shows with information below

apiVersion: v1

clusters:

- cluster:

certificate-authority-data: DATA+OMITTED

server: https://aks-cluste-aks-rg-3f2c36-a86fa58c.hcp.eastus.azmk8s.io:443

- cluster:

certificate-authority-data: DATA+OMITTED

server: https://kubernetes.docker.internal:6443

- cluster:

certificate-authority-data: DATA+OMITTED

server: https://kuar-clust-kuar-3f2c36-3176b3bf.hcp.eastus.azmk8s.io:443

- cluster:

certificate-authority: C:\Users\admin\.minikube\ca.crt

server: https://192.168.2.14:8443

- cluster:

certificate-authority-data: DATA+OMITTED

server: https://myaksclust-myresourcegroup-3f2c36-40a876eb.hcp.eastus.azmk8s.io:443

contexts:

- context:

cluster: aks-cluster

user: clusterUser_aks-rg_aks-cluster

- context:

cluster: docker-desktop

user: docker-desktop

- context:

cluster: kuar-cluster

user: clusterUser_kuar_kuar-cluster

- context:

cluster: minikube

user: minikube

- context:

cluster: myAKSCluster

user: clusterUser_myResourceGroup_myAKSCluster

current-context: myAKSCluster

kind: Config

preferences: {}

users:

- name: clusterUser_aks-rg_aks-cluster

user:

client-certificate-data: REDACTED

client-key-data: REDACTED

token: e4d785d3065d5697f03ac404ac27460f68b973b7185ea4697679077e39ff106437d3a7fe9c79f3be5cc84bbab6728e2d71bc010d0a6c6628bf05162958c122bb

- name: clusterUser_kuar_kuar-cluster

user:

client-certificate-data: REDACTED

client-key-data: REDACTED

token: afc713faf0d9c13f494711c3ab569d52c48c4ac1bef72f53329ab832247c0180f87a167d59c1a784ea0951bd11f043e8eb8aa6089caec2d9d71f61ea58d29b48

- name: clusterUser_myResourceGroup_myAKSCluster

user:

client-certificate-data: REDACTED

client-key-data: REDACTED

token: 2906f9edf219806a855010df4aa104f25eceea94ce748cf1b7813f3af93c531a60b853654b6bc0b24c448491ab3a5896ad36e0fa00d898507c44db7c69f8b283

- name: docker-desktop

user:

client-certificate-data: REDACTED

client-key-data: REDACTED

- name: minikube

user:

client-certificate: C:\Users\admin\.minikube\profiles\minikube\client.crt

client-key: C:\Users\admin\.minikube\profiles\minikube\client.key

we can see that the current-context is myAKSCluster, which is Azure Kubernetes Service. the error is sterm from that either authoriation token had been expired or the service instance had been removed from azure

Since we already know the root cause of the issue, we can change the kubectl context to the new AKS or the local docker-desktop.

we can follow the steps below

#to get all the context available in the kubectl configuration file

kubectl config get-contexts

CURRENT NAME CLUSTER AUTHINFO NAMESPACE

aks-cluster aks-cluster clusterUser_aks-rg_aks-cluster

docker-desktop docker-desktop docker-desktop

kuar-cluster kuar-cluster clusterUser_kuar_kuar-cluster

minikube minikube minikube

* myAKSCluster myAKSCluster clusterUser_myResourceGroup_myAKSCluster

#update the current context with local docker-desktop

kubectl config use-context docker-desktop

Wednesday, July 14, 2021

how to pass dockerfile argument from docker-compose file?

It should be very convenient and dynamic, if we can set our environment variable on docker build time.

since I want to turn graphql playground on and off during the production deployment, I already use the

NODE_ENV to control the playground feature in the .env file.

in .net file

NODE_ENV=dev

GraphQLModule.forRoot({
      playground: process.env.NODE_ENV === 'dev',
      debug: process.env.NODE_ENV === 'dev',
      autoSchemaFile: true,
    }),

we can use ARG to set a variable in the dockerfile that we can pass in during docker build run.

ARG node_env_var
ENV NODE_ENV=$node_env_var
RUN echo $node_env_var

here is the full content of Dockfile

FROM node:14 as production

ARG node_env_var
ENV NODE_ENV=$node_env_var
RUN echo $node_env_var

WORKDIR /usr/src/api

COPY package.json .
COPY yarn.lock .

RUN yarn global add @nestjs/cli

RUN yarn install --production=true

## install NIC s a computer networking utility for reading from and 
# writing to network connections using TCP or UDP.
RUN apt-get -q update && apt-get -qy install netcat

COPY . .

RUN yarn build

CMD ["sh", "-c", "yarn typeorm migration:run && yarn start:prod"]

in the Docker-compose.yml file we can set the args variable. when I run the deployment in the production environment, i can simply change the node_env_var: production

build: 
            context: server
            target: production
            dockerfile: Dockerfile
            args:
                node_env_var: dev

here is the docker-compse.yml file

version: "3.8"

services:
    mysqldb:
        image: mysql
        environment: 
            - MYSQL_ROOT_PASSWORD=Mys0l@123456
            - MYSQL_DATABASE=hyrecar
        ports: 
            - 3306:3306
        command: --default-authentication-plugin=mysql_native_password
        networks:
            - shared-network
        volumes: 
            - db-config:/etc/mysql
            - db-data:/var/lib/mysql
            - ./db/backup/files:/data_backup/data
       
    #acting as a proxy

    nginx:
        image: nginx:latest
        container_name: nginx-prod
        volumes: 
            - ./nginx/nginx.conf:/etc/nginx/nginx.conf
        ports: 
            - 81:80
        command: /bin/sh -c "nginx -g 'daemon off;'"    
        depends_on: 
            api-prod:
                condition: service_healthy
            app-prod:
                condition: service_started
        networks: 
            - shared-network

    api-prod:
        container_name: nestjs_api_prod
        image: nestjs_api_prod:1.0.0.0
        build: 
            context: server
            target: production
            dockerfile: Dockerfile
            args:
                node_env_var: dev
        command: sh -c './bin/wait-for -t 0 mysqldb:3306 -- yarn start:prod'
        depends_on: 
            - mysqldb
        networks: 
            - shared-network
        ports: 
            - 9000:9000
        restart: unless-stopped
        healthcheck:
            test: ["CMD", "curl", "http://api-prod:9000"]
            interval: 5s
            timeout: 3s
            retries: 6     
        
    app-prod:
        container_name: rect_app_prod
        image: rect_app_prod:1.0.0
        build:
            context: web
            target: production
            dockerfile: Dockerfile
        command: yarn run start:prod
        ports: 
            - 3000:3000
        networks: 
            - shared-network
        restart: unless-stopped

#intialize network
networks: 
    shared-network:

#Initializing docker compose volumes
volumes: 
    db-config:
    db-data:

for development and test environment deployment.

set node_env_var: dev in docker-compose.yml

run docker-compose build to rebuilt the container

execute docker-compose up to run all containers

in the production, we just need to change node_env_var: production

then we can run docker-compose build & docker-compose up to turn off graphql playground feature for security reason.

Please Note you can not simply update the variable in the docker-compose yaml file and run docker-compose up since the variable is passing in at docker-compose build time, not the run time. so We have to run docker-compose build process to take the effect.

Monday, February 24, 2020

A cool tool to validate the yaml content on building the docker-compose.yml file.

after i added a docker-compose project the asp.net solution, then i try to build the docker-compose file to generate all the images.

however i always encounter error without specific message. the root of cause this issue is that
YAML follows indentation structure very strictly. one extra space or tab will cause this issue.

this online tool will be a great help on validation your content in YAML.

https://onlineyamltools.com/validate-yaml

here is my sample docker-compose file, after i pasted it in the yaml box, the line causes the issue highlights immediately in the errors box

version: '3.7'

services:
webapi_dotnetcore:
image: ${DOCKER_REGISTRY-}webapidotnetcore
build:
context: .
dockerfile: WebAPI_DotNetCore/Dockerfile

db:
image: mcr.microsoft.com/mssql/server
environment:
- ACCEPT_EULA=Y
- SA_PASSWORD=Password_01
ports:
- '1433:1433'

Sunday, February 23, 2020

How to fix COPY failed COPY failed: stat /var/lib/docker/tmp/docker-builder/appFolder/App.csproj: no such file or directory issue in Docker Build command to create a docker image in Microservice project?

I am playing with the Microservices application with docker enable in my local development.

after i completely setup my project. i try to manipulate the project with docker building command.

since the dockerfile had been automatically generated by visual studio, when i create a dotnet core web api project with docker support.

how ever when i run this command to create a docker image

docker build . -t myimage -f dockerfile

it encountered an error on the copy step. i put the step with error in bold.

FROM mcr.microsoft.com/dotnet/core/aspnet:2.1-stretch-slim AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443

FROM mcr.microsoft.com/dotnet/core/sdk:2.1-stretch AS build
WORKDIR /src
COPY ["simplemicroservicesApp/simplemicroservicesApp.csproj", "simplemicroservicesApp/"]
RUN dotnet restore "simplemicroservicesApp/simplemicroservicesApp.csproj"
COPY . .
WORKDIR "/src/simplemicroservicesApp"
RUN dotnet build "simplemicroservicesApp.csproj" -c Release -o /app/build

FROM build AS publish
RUN dotnet publish "simplemicroservicesApp.csproj" -c Release -o /app/publish

FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "simplemicroservicesApp.dll"]

here is the error message

Step 7/17 : COPY ["simplemicroservicesApp/simplemicroservicesApp.csproj", "simplemicroservicesApp/"]
COPY failed: stat /var/lib/docker/tmp/docker-builder663072116/simplemicroservicesApp/simplemicroservicesApp.csproj: no such file or directory

the root of cause of this issue is that Visual Studio generated the dockerfile inside the project folder. actually it should be in the solution folder.

by default the dockerfile was generated inside the project folder

actually it should be in the solution level in order to build a docker image properly.

after i cut and paste the file into the solution folder, then navigate to the solution folder in window CMD, and execute the following line to create a docker image.

docker build . -t myimage -f dockerfile

Thursday, February 6, 2020

how to resolve the docker building issue "dial tcp: lookup auth.docker.io on 192.168.65.1:53: no such host"?

i try to play with a very simple script to build a docker from my local machine

the error came up immediately with following message

//Command to be execute to build a docker
λ docker build . -t hostname
Sending build context to Docker daemon 4.608kB
Step 1/5 : FROM nginx:latest
Get https://registry-1.docker.io/v2/library/nginx/manifests/latest: Get https://auth.docker.io/token?scope=repository%3Alibrary%2Fnginx%3Apull&service=registry.docker.io: dial tcp: lookup auth.docker.io on 192.168.65.1:53: no such host

The root cause of this issue is that though the Docker Desktop is running in my local, however i did not log into the docker hub.

after i log into the docker and run the command again. the result is succeed.

Sending build context to Docker daemon 4.608kB
Step 1/5 : FROM nginx:latest
latest: Pulling from library/nginx
bc51dd8edc1b: Pull complete
66ba67045f57: Pull complete
bf317aa10aa5: Pull complete
Digest: sha256:ad5552c786f128e389a0263104ae39f3d3c7895579d45ae716f528185b36bc6f
Status: Downloaded newer image for nginx:latest
---> 2073e0bcb60e
Step 2/5 : ARG VERSION
---> Running in 75e8c63f7258
Removing intermediate container 75e8c63f7258
---> 74f3c847d731
Step 3/5 : ENV VERSION=$VERSION
---> Running in 6d1ef8500966
Removing intermediate container 6d1ef8500966
---> 3de0be6fa8a0
Step 4/5 : COPY hostname.sh .
---> 451b37ea6530
Step 5/5 : CMD ["/hostname.sh"]
---> Running in b71381f7ba23
Removing intermediate container b71381f7ba23
---> 47870a87b284
Successfully built 47870a87b284
Successfully tagged hostname:latest