Deploying Using Containers

Owned by blake.wilkinson
Last updated: Apr 18, 2024
14 min read

Containers represent a repeatable, immutable, and portable way to run software on-premise or in the cloud. Coupled with Container orchestration, it means an opportunity to deploy software in a highly resilient, and scalable fashion. At Panintelligence, we offer our software in containers to facilitate your stable deployment of our software.

Migrating from all-in-one, Marialess, and other deployments of Panintelligence

What are microservices

Traditionally built software required components to be deployed next to each other with mutual dependencies. If there were to be a failure with any part of that software, or if a package needed to be updated that caused issues with other parts of the software, this would create difficulties. Containers represent the opportunity to create scope by which smaller components for software that can still talk to each other, can be deployed in a way that no longer is mutually reliant on other functions. Pi is designed this way.

5 distinct microservices form the architectural blend for Pi. These are:

Dashboard

This is the nucleus of the pi infrastructure. Nothing much will work without this.

Scheduler

This handles the report generation and task scheduler for the dashboard.

Analytics

This is the analytics engine for the dashboard, formally known as Pirana. This microservice is fundamental to forming the mushroom maps in the dashboard.

Renderer

This is the service that takes HTML charts and renders them into whatever shape the end user requires. The more resources you can dedicate to the renderer, the better. Fewer resources means slower renders.

excel-reader

This stages an Excel file into a SQLite database for consumption with the dashboard.

why would I want to use separate containers?

Some elements of any software solution are more heavily accessed than others. By breaking it into smaller chunks, you have better control over scaling each part. Additionally, in the event of a failure, you don’t lose access to other functioning areas of the software, as you would with a monolith. Likewise, a misbehaving component will not collapse your entire stack. It also permits you to monitor individual components more closely for failures and develop automated recovery plans.

how do I migrate from a single container-based installation to separate containers?

Database

you will need to be able to execute mysqldump and mysql commands against your database. You could install MySQL locally, or you could make use of docker by running the following command on your local machine.

docker run -dt mariadb:latest database docker exec -it database bash

This will put you into an interactive session on the container which will have the SQL tools you need to effect a backup and restore. Please note that container filesystems are immutable. If you remove your container, any extract you’ve taken will be lost, unless you copy it to your host filesystem. You can copy files from your container by following this set of instructions.

Panintelligence uses a repository database containing metadata on defining elements in the dashboard such as categories, charts, users, user roles and permissions, and audit information. To back this up, you will need to perform the following action:

mysqldump \ --add-drop-table \ --add-drop-database \ --databases \ -u{db_username} \ -p{db_password} \ -h{db_host} \ -P{db_port} \ {db_schema_name} \ --ignore-table={db_schema_name}.mis_user_cat_access_view_pi \ --ignore-table={db_schema_name}.test_user_access \ --ignore-table={db_schema_name}.mis_user_cat_access_view \ > sqldump{datenow}.sql

You will then need to insert this data into your new database. be mindful, that if you have already created your database by launching Panintelligence and are trying to overwrite, you will need to stop your dashboard server. This is because there are table locks generated by the dashboard application which will prevent the drop action of the SQL script you generated when creating your backup.

mysql \ -u{db_username} \ -p{db_password} \ -h{db_host} \ -P{db_port} \ {db_schema_name} < {yoursqlfile}.sql

Persistent items

you will now want to take a backup of the following items and mount the volumes to your new deployment. you can find information about docker volumes here.

  • Themes

    • ${source_dir}/Dashboard/tomcat/webapps/panMISDashboardResources/themes

  • Images

    • ${source_dir}/Dashboard/tomcat/webapps/panMISDashboardResources/images

  • Custom JDBC

    • ${source_dir}/Dashboard/tomcat/custom_jdbc_drivers

  • SVG

    • ${source_dir}/Dashboard/tomcat/webapps/panMISDashboardResources/svg

 

Quick Start

Links here to a docker-compose including MariaDB container which is our reference deployment with only our software + maria (i.e. the most straightforward setup utilizing multiple containers).

Common Deployment Pattern

AWS - ECS

terraform

Deleteing EFS

  • remove the EFS backup vault

  • tear down using terraform scripts

  • remove s3 terraform state files and bucket

Resilience

It’s a good idea to deploy your dashboard across more than one availability zone and also to employ an auto-scaling group on the analytics, renderer, and dashboard services. You must only run a single Scheduler task. The dashboard is based on Java, which ringfences memory, as a result, you must use CPU as the value to trigger scaling.

Container insights

In order to gain more insight into the performance of your instances running in an ECS cluster, it will be important to enable “container insights”. Be aware, there is an additional cost. You will receive basic telemetry on your service performance without container insights.

Logs

you must enable a log group for each of your task definitions to ensure you receive logs. Without this, the logs from the container will not be captured. As a prerequisite, you will need to create the custom log group before defining this in your task definition.

 

Kubernetes

Persistence

Panintelligence requires some files to be shared amongst pods. to achieve this end, you will need to create a persistent volume. This must be of a storage Capacity of at least 5Gi, however it’s advised to give some space to grow (should you wish to include more custom jdbc, images or themes). an example is shown below

For simplicity, you can choose to copy this to a file pi-pv.yaml and execute kubectl apply -f pi-pv.yaml to create the persistent volume for use by the panintelligence deployment in your cluster.

Secrets

To access panintelligence containers, we will need to whitelist your user. Please contact your CSM who will grant you this access. you will then need to create a secret which is to be used by the helm charts.

Helm

some helm commands go here

Ingress

here is an example ingress for the dashboard.

Swarm

put an example of a swarm deployment here.

Azure

Azure allows for multiple deployment methodologies.

App Service

As previously stated, Panintelligence can be deployed as distinct web applications, which allows you to choose which components you wish to deploy based on your requirements, tailoring your deployment plan to your current needs.

execute the following statements:

Prerequisites

Please run to set these environment variables before executing any other scripts.

Dashboard

please run the prerequisites before running this script in the AZ cli.

Renderer

please run the prerequisites before running these scripts.

After your service is verified as being available, you will need to set up the renderer by logging into the dashboard, and then setting the PAN_RENDERER_URL to the URL of the renderer you’ve just deployed. it should look something like “pan<your environment name><environment>renderer”. If you apply a custom domain, this will change.

Scheduler
Analytics

Pulling the images to a local repository

We recommend not relying on a third party to host your containers for your business continuity. Therefore we highly recommend pulling, retagging, and then pushing your containers to your private image repository.

We’d also advise against using the “latest” tag in production but use specific versions so you can control when and how you migrate to the latest version of our software.

Healthchecks

how do we know the container is functioning? we use a health check! for orchestration, you can use the following endpoints that will all return a http status code of 200 when the software is functioning well:

Server

Renderer

Scheduler

Pi Analytics

Configuring your containers

Containers are immutable. If a container stops working (determined by the healthcheck) then you will want to start another container immediately. some configuration will need to be persisted using environment variables, and some using volumes

Environment Variables

Our example scripts below show the minimum environment variables you will need to get up and running, there are however many other options which are listed on our Environment Variables documentation page

Example of a standard environment configuration

variable

description

example value

variable

description

example value

PI_DB_HOST

the database host that’s hosting the repository database for the dashboard

mypidatabase.mydns.com

PI_DB_PASSWORD

The database password for the database that’s hosting the repository for the dashboard

my5up3r53cur3P4$$w0rd!

PI_DB_USERNAME

The database username for the database that’s hosting the repository for the dashboard

megadbusername

PI_DB_SCHEMA_NAME

The database schema for the database that’s hosting the repository for the dashboard. default dashboard

myawesemedashboardrepo

PI_DB_PORT

database for for the repository database

3306

PI_EXTERNAL_DB

sets the database to use an external repository as opposed to a supplied internal repository (note with the “server” container, there is no supplied db)

“true”

PI_LICENCE

your dashboard licence as supplied by your CSM

 

PI_TOMCAT_MAX_MEMORY

The max memory that is allocated to the java heapspace for the application. N.B. when defining this, you will need to leave the recommended allowance for the OS. for our containers, this is 1GB

2048

PI_PROXY_ENABLED

sets the dashboard configuration to understand that it is behind a proxy.

“true”

PI_PROXY_SCHEME

This is used to dress the redirects with the correct scheme (HTTP / HTTPS)

HTTPS

PI_PROXY_HOST

This is used to dress the redirects with the correct URL

spankingdashboard.com

PI_PROXY_PORT

this is used to set the redirect port

443

PI_TOMCAT_MAX_TOTAL_DB_CONNECTIONS

the max number of concurrent connections the database can hold open to the repository database (be warned, most databases set a small limit for this that will need increasing.)

100

PI_PROXY_IS_SECURE

determines whether the proxy is TLS/SSL enabled

“true”

PI_TOMCAT_COOKIE_NAME

if your users are going to be connecting to more that one instance of pi at the same time, it’s a good idea to set this to something unique for each deployment.

PIISGREAT

PI_TOMCAT_FRAME_ANCESTORS

this is a space separated whitelist of domains from which you permit embedding in iframes.

https://mycoolwebsite.com

RENDERER_DASHBOARD_URL

renderer will need to find your dashboard installation. this value will be passed to the renderer from the dashboard. It should be the full path to your dashboard.

https://spankingdashboard.com/pi

Volumes

Certain items used by Panintelligence software need to be stored on a file system (these will be read-write items). Below is a list of the volumes you may need to setup and what they are used for. Our example scripts always include these.

Dashboard

Themes

Internal Location: /var/panintelligence/Dashboard/tomcat/webapps/panMISDashboardResources/themes

Use: Custom UI themes for Panintelligence software

Images

Internal Location: /var/panintelligence/Dashboard/tomcat/webapps/panMISDashboardResources/images

Use: custom images

Keys

Internal Location: /var/panintelligence/Dashboard/keys/

Use: Generated keys used for communication between dashboard and scheduler services

Custom JDBC

Internal Location: /var/panintelligence/Dashboard/tomcat/custom_jdbc_drivers

Use: Additional JDBC drivers that are not bundled, used for free-format jdbc connections

SVG

Internal Location: /var/panintelligence/Dashboard/tomcat/webapps/panMISDashboardResources/svg

Use: SVG images for custom maps functionality

Specifications

Server

Item

Value

Item

Value

CPU

1

Ram

2048MB

Min Count

1

Desired Count

3

Notes

We’ve stress-tested this combination to 45 concurrent users. The simulation for the test included the simulant users making constant requests to the dashboard. This is heavy, almost excessive use of the dashboard, and not precisely the usage patterns you’d find from a user. As a result, under real-world conditions, this configuration will likely support many more users.

Renderer

Item

Value

Item

Value

CPU

2

Ram

4048MB

Min Count

1

Desired Count

1

Notes

If you’re rendering in “real time” then you will want to give the renderer as much resource as you can spare. This is because the more resources you give the renderer, the faster the results.

Scheduler

Item

Value

Item

Value

CPU

0.5

Ram

1024MB

Min Count

1

Max Count

1

Analytics

Item

Value

Item

Value

CPU

0.25

Ram

512MB

Min Count

1

Desired Count

1

Examples

We offer example scripts of our recommended way to deploy using containerisation. There is flexibility in how the software can be deployed however we suggest using these as a starting point and these examples are what we are best placed to support.

  • Docker-compose with Maria DB

  • Docker-compose with external Maria DB

  • Others…