...
We host the container images for panintelligence at github container registry (GHCR). In order to access our images, please create a support request with our team at support@panintelligence.com providing your github username or email. The team will respond with a list of images to which you have been granted access.
It’s highly recommended and endorsed that you pull our images and push to your own private image repository so you can maintain service continuity without relying on a third party.
Terraform
Please review the documentation provided by Hashicorp for the most comprehensive and up to date documentation pertaining to installation of Terraform on your chosen platform https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli
...
it’s recommended to leave around 1Gb storage for images, themes, “SVG” files, custom JDBC and locale information. This will flex based on your own requirements and is dependent on the size of the images you use. Since Pi is a web-based system, it’s recommended that you compress images to the smallest storage requirements while maintaining image quality to facilitate transfer speeds.
Security
Panintelligence
Public Access
The dashboard container is accessible through port 443 via the load balancer.
...
Private Access
There is no direct access expressly added to the repository database. access to the database can be conducted via adminer. If
AWS RDS MariaDb
AWS EFS
EFS is encrypted at rest.
...
There is one lambda function that configures the EFS. This lambda function only has access to the EFS volume and MUST only be executed once. repeated execution of this lamdba function will overwrite resources in the EFS, eliminating all changes.
Multi AZ deployment
Auto-scaling deployment
The dashboard autoscaling is set initially for a min and max capacity of 1. This should be set higher should you wish to enable autoscaling. the policy is set to scale based on a CPU exhaustion.
because the dashboard container runs using tomcat, you should not set the autoscaling rules to use memory as this would cause the dashboard to prematurely scale out, and never back in.
Persistent Storage on AWS EFS
Database Respository (metadata) on AWS RDS
Model | Core Count | vCPU* | CPU Credits/hour | Memory (GiB) | Network Performance (Gbps) |
|---|---|---|---|---|---|
db.t3.micro | 1 | 2 | 12 | 1 | Up to 5 |
db.t3.small | 1 | 2 | 24 | 2 | Up to 5 |
Panintelligence
Obtaining a licence key
To obtain your licence, please contact your CSM at panintelligence. If you’re unaware who your CSM is, please contact the support desk support@panintelligence.com and your request will be appropriately routed.
Automated Licence Manager
those looking to authenticate their product using the ALM must place the customer key against the PI_LICENCE environment variable on the dashboard service. This service must have the capability to talk out to the internet. when the dashboard first starts, the process will reach out to our system and obtain the licence pertaining to your customer ID. In the event of an unsuccessful initial callback, you will receive a invalid licence notification.
Offline Licencing
Customers who are unable to provision a system that has egress access to the internet from the application layer may be delivered an offline licence. This licence contains everything required to fully licence panintelligence. This licence is non transferrable. A new licence for each deployment should be sought from your CSM. The licence should be stripped of all carriage returns and line spacing and placed inline in the PI_LICENCE environment variable. Please note that these licences are time limited and thus represents and operational overhead to ensure the licences are kept up to date. Also these licences are of fixed scope. If you require more users or additional modules, you will contact your CSM who can create you a new licence.
Key Skills
Terraform
AWS VPC (security groups, networking, subnets, nacls, route tables)
AWS Lambda
AWS ECS
Containerisation
MySQL
AWS EFS
AWS Cloudwatch
AWS Load Balancing (Application Load Balancing)
AWS IAM
AWS Route53
AWS Secrets Manager
SSL/TLS Certificates
Horizontal Scaling
Linux
Key Run Information
Terraform
Although the terraform scripts are self documenting, it helps to become familiar with the HCL notation that hashicorp uses when provisioning systems with terraform. You can find excellent documentation for your first steps into terraform by following the link: https://developer.hashicorp.com/terraform/intro
Networking and security groups
The panintelligence deployment detailed in the terraform scripts makes use of VPC, subnets, security groups and other AWS assets in order to build out a scalable deployment of panintelligence. The code is reuseable and parameterisable. You’re encouraged to fork from our project. We also welcome contributions from our community for improvements to our scripts.
Backup and Restore
Database Disaster Recovery
by default, the database retention period is 3 days. Should you need to restore from a backup, you should follow the AWS backup restore process https://docs.aws.amazon.com/aws-backup/latest/devguide/restoring-rds.html
Please note that this will restore to an entire new database. you will need to connect to this database using an administration tool, migrate your backed up database, then issue a restoration.
one approach to this is to launch an interactive container or ec2 instance with mutual access to both databases. install a mariadb client, then follow the following steps to extract and import your database
backup extract (source database)
| Code Block |
|---|
mysqldump \
--add-drop-table \
--add-drop-database \
--databases \
-u<admin_user> \
-p<your password> \
-h<database source host endpoint> \
-P3306 \
dashboard \
--ignore-table=dashboard.mis_user_cat_access_view_pi \
--ignore-table=dashboard.test_user_access \
--ignore-table=dashboard.mis_user_cat_access_view \
> sqldump_dashboard<todays_date>.sql |
backup restore (target database)
| Code Block |
|---|
mysql \
-u<db_user> \
-p<your password> \
-h<database target host endpoint> \
-P3306 < sqldump_dashboard<todays_date>.sql |
AWS EFS Disaster Recovery
EFS is backed up using EFS backup vault. please follow the AWS instructions to effect a restore
https://docs.aws.amazon.com/aws-backup/latest/devguide/restoring-efs.html
Time To Deploy / Restore
The script takes approximately 20 minutes to run.
Restoring a database using the delta backup routine from aws takes approximately 20 minutes with additional time required to move relevant extracts around. If you have access set up, the whole process should take no longer than 30 minutes.
Healthchecks, Logging, Troubleshooting
Target Groups
AWS Cloudwatch
How to check if your Panintelligence dashboard is healthy? How can you troubleshoot if there are issues?
| Info |
|---|
Metrics to look out for: You want to see if there is any “unhealthy hosts”, if there is any, that’s a sign there is a problem. Look into that particular EC2 instance to find out. |
Target groups:
Log into AWS Management console → Go into EC2 console → On the left side, click into the ‘Target Groups’ tab
Click into the Panintelligence Target:
The image below shows where to check for any unhealthy targets
If you do see any unhealthy targets, you can check the ‘Monitoring’ tab to look at the metrics for more details at a specific time
| Info |
|---|
If you face a ‘502’ error on the front end, check the target groups. It may be unhealthy or there is no targets registered to the AWS Application Load Balancer. |
AWS Cloudwatch
Accessing your instance for administration
Because of the nature of containerisation, changes to the container should not be performed interactively or directly. You can set most settings using environment variables for which there is a comprehensive list available here.
You are welcome to make changes to the base container by means of a Dockerfile if you wish to add additional software or packages. Please note that changes made to the base container will fall outside of support. having said that, we’re only too happy to help if you have found an issue. You can obtain assistance by reaching out to our support team at support@panintelligence.com
AWS RDS MariaDB to check
...
if the Panintelligence Repo is healthy:
| Info |
|---|
Metrics to look out for:The “CPU Utilization” metric is watch due to how well the database is handling the workload. If it’s near 90%. Look into why there is a huge workload or increase the RDS instance type. In addition, you are limited to “DB Connections”, if it’s near 300 then you’re near the limit. Look into reducing the connections or use RDS proxy to assist. |
AWS RDS MariaDB: To view the logs, please follow these instructions linked here:
Log into AWS Management console → Go into AWS RDS console → Click into ‘Databases’ → Click into ‘dashboard’:
You can check the ‘Monitoring’ tab. This is to see if you are hitting max connections, the CPU Utilisation and memory space :
You can even go further and check the RDS MariaDB Logs, select the logs near the bottom and select ‘View’ as highlighted below:
Cost Estimates
How to Obtain Support assistance
Our team are on hand during UK office hours and can be raised by emailing support@panintelligence.com
If you require assistance with Panintelligence to help with visualisations, it may be advantageous for you to book time with one of our consultants. Please reach out to your CSM who will schedule this with you. They can be raised at csm@panintelligence.com