Introduction
Panintelligence is the easy-to-use and quick-to-deploy solution to unlocking powerful data insights. We're the only software solution on the market that brings together business intelligence and machine learning to a new self-service level, empowering you to gain full control over your data.
Access unprecedented styling options to make our software look just like your own, and hook up cloud data warehousing and ETL (Extract, Transform and Load) tools, as well as your core product, to create a truly seamless analytics experience.
Simply connect your databases to the Panintelligence dashboard. The Panintelligence dashboard only has read access so it won’t modify any of your data.
Panintelligence does not collect your data or move it.
Panintelligence does not require root/admin access on the server. Sign in as “pi-user”.
This project supports multi-availability zone deployment. For more details please take a look at System architecture and Multi-Availability zone deployment. In addition, the monitoring and logging will also explain in more detail.
Installation of Key Tools
Terraform
Please review the documentation provided by Hashicorp for the most comprehensive and up to date documentation pertaining to installation of Terraform on your chosen platform https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli
System Architecture
Diagram
Resources
Resource | Description | How is it used? |
---|---|---|
Route 53 | Amazon Route 53 is a highly available and scalable cloud Domain Name System (DNS) web service. For more information, please see Setting up Amazon Route 53 documentation | You can attach your domain name to the AWS Application Load Balancer to point to the Panintelligence dashboardAWS ACM |
AWS ACM | AWS Certificate Manager is a service that lets you easily provision, manage, and deploy public and private Secure Sockets Layer/Transport Layer Security (SSL/TLS) certificates for use with AWS services and your internal connected resources. For more information, please see Setting up - AWS Certificate Manager documentation | In order to use port 443/HTTPS in the AWS Application Load Balancer, you will need an SSL certificate. |
AWS S3 Bucket | Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance. For more information, please see AWS S3 Bucket documentation | The architecture requires the user to upload a lambda zip provided in the Git repository and another s3 bucket is created to store images, themes and excel-data. |
AWS Internet gateway | An internet gateway is a horizontally scaled, redundant, and highly available VPC component that allows communication between your VPC and the internet. For more information, please see Internet Gateways documentation | The Panintelligence dashboard requires web browser access. |
AWS IAM | AWS Identity and Access Management (IAM) enables you to manage access to AWS services and resources securely. Using IAM, you can create and manage AWS users and groups, and use permissions to allow and deny their access to AWS resources. For more information, please see AWS IAM documentation | IAM permissions allows you to have fine grain control on who and what has access to resources. |
AWS Security groups | A security group acts as a virtual firewall for your instance to control inbound and outbound traffic. For more information, please AWS Security Groups documentation | Increase protection to your infrastructure. |
AWS Application Load Balancer | Elastic Load Balancing automatically distributes your incoming traffic across multiple targets, such as EC2 instances, containers, and IP addresses, in one or more Availability Zones. It monitors the health of its registered targets, and routes traffic only to the healthy targets. For more information, please see AWS Application Load Balancer documentation | The ALB directs traffic to the healthy EC2 targets. |
AWS VPC | Amazon Virtual Private Cloud (Amazon VPC) enables you to launch AWS resources into a virtual network that you've defined. This virtual network closely resembles a traditional network that you'd operate in your own data centre, with the benefits of using the scalable infrastructure of AWS. For more information, please see AWS VPC documentation | We use the AWS VPC to launch resources in the virtual network. |
Subnets | You need to specify a logical address to specific resources. For more information, please see Subnets documentation | Configure resources to specific subnet cidr blocks. |
NACL | A network access control list (ACL) is an optional layer of security for your VPC that acts as a firewall for controlling traffic in and out of one or more subnets. For more information, please see NACL documentation | Configure additional security. |
AWS Lambda | AWS Lambda is a serverless compute service that lets you run code without provisioning or managing servers, creating workload-aware cluster scaling logic, maintaining event integrations, or managing runtimes. For more information, please see AWS Lambda documentation | The infrastructure uses AWS Lambda to side load S3 objects to AWS EFS |
AWS RDS MariaDB | Amazon Relational Database Service (Amazon RDS) is a web service that makes it easier to set up, operate, and scale a relational database in the AWS Cloud. For more information, please see AWS RDS MariaDB documentation | The Panintelligence dashboard uses AWS RDS MariaDB as an external DB. |
AWS EFS | Amazon Elastic File System (Amazon EFS) provides a simple, serverless, set-and-forget elastic file system for use with AWS Cloud services and on-premises resources. For more information, please see AWS EFS documentation | AWS EFS is used to keep persistent data for themes, images, SVG and custom jdbc |
AWS Auto scaling | AWS Auto Scaling monitors your applications and automatically adjusts capacity to maintain steady, predictable performance at the lowest possible cost. For more information, please see AWS Auto scaling documentation | Auto scaling is used to increase or decrease the EC2 instances depending on traffic. |
AWS NAT gateway | A NAT gateway is a Network Address Translation (NAT) service. You can use a NAT gateway so that instances in a private subnet can connect to services outside your VPC but external services cannot initiate a connection with those instances. Please see AWS NAT gateway documentation | Allows you to use Panintelligence Automated Lincence Manager |
AWS ECS | Amazon Elastic Container Service (ECS) is a fully managed container orchestration service that simplifies running, scaling, and securing Docker container applications on AWS. It integrates seamlessly with other AWS services. | manages the containers in the microservice modelA |
AWS Cloudwatch | Amazon CloudWatch is a monitoring and observability service designed for DevOps engineers, developers, and IT managers. It provides real-time monitoring, logging, and alarms for AWS resources, applications, and services, ensuring operational health and performance optimization. | stores logs and container metrics and insights for monitoring, logging and scaling purposes. |
Service Quotas
AWS Resource | Service quotas that the infrastructure uses | Notes |
---|---|---|
AWS ECS (cluster) | 1 | |
AWS ECS (task) | 5 - 8 | Scaling policies will dictate how many tasks are in operation |
AWS ECS (Service) | 5 | |
Target groups per Auto Scaling group | 2 | |
AWS Auto scaling per region | 2 | |
AWS VPC | 1 | VPC configuration is conducted outside the main terraform script |
AWS Internet gateway | 1 | |
General Purpose SSD (gp2) volume storage | 200 | |
AWS Application load balancer | 1 | |
AWS Lambda Function time out | 15 | |
AWS Lambda temporary storage | AWS Default quota value is 512MB, A Panintelligence theme or image would not hit the limit. | |
EFS per VPC | 1 | |
EFS Mount targets | 2 | |
EFS attached security group | 1 | |
Interface VPC endpoints per VPC | 8 | |
Route tables per VPC | n/a | |
Routes per route table | 10 | |
Subnets per VPC | 6 | |
Security groups | 10 | |
Network ACLs per VPC | 10 | |
Secrets per account | 1 | |
DB instance | 1 | |
DB Parameter group | 1 | |
DB subnet group | 1 | |
DB security group | 1 | |
AWS NAT Gateway | 1 |
Technical Data sheet requirements
The technical datasheet offers some guidance on how much resources you would require depending on your infrastructure and users.
Please take a look at this link.
Sizing and recommendations
Service | CPU | Memory | Storage | Notes |
---|---|---|---|---|
Dashboard | 1 | 2Gb | 100Gb | |
renderer | 2 | 4gb | The more resource you give, the faster resources render. This is only for ad-hoc request or reports. | |
pirana | 0.5 | 512mb | ||
scheduler | 0.5 | 512mb | ||
Storage requirements
Database storage
EFS storage
it’s recommended to leave around 1Gb storage for images, themes, “SVG” files, custom JDBC and locale information. This will flex based on your own requirements and is dependent on the size of the images you use. Since Pi is a web-based system, it’s recommended that you compress images to the smallest storage requirements while maintaining image quality to facilitate transfer speeds.
Data security
AWS RDS MariaDb
AWS EFS
EFS is encrypted at rest.
AWS IAM
The terraform scripts follow least possible permissions where possible.
AWS Lambda
There is one lambda function that configures the EFS
Add Comment