Appanvil karma designer | ||||
---|---|---|---|---|
|
Table of Contents | ||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Customer Request tickets are highlighted with the 🗣️ icon
The 2024_09 release contains x1 Schema change
IMPORTANT: New Minimum Server Requirements
Info |
---|
IMPORTANT: As a precaution, we removed the ability to download the September 24 release whilst we investigated a couple of issues. To take advantage of the changes listed in this release note, you will need to upgrade to the November 24 release. |
Note |
---|
N.B. We have revised the minimum server requirements for a production system node. This also includes a recommended change to environment variables - see HERE for more information. |
NEW FEATURES
Custom Fields (BETA) 🗣️ | - | 29554 | Custom Fields | ||
---|---|---|---|---|---|
We’ve added an innovative way to unlock key-value data in your databases which has traditionally been difficult to report on. Known as Custom Fields, you can now provide additional configuration to have the Dashboard create objects automatically from the key-value data where previously you would have needed to manually create these for each item. Click HERE for more information on the Custom Fields feature.
|
Single Chart Word Export - Performance Improvement (BETA) 🗣️ | pi#2438 | 29822 & 32731 | Exporting - Word |
---|---|---|---|
We’re excited to offer a completely rebuilt Word (docx) export functionality which is almost instant and reflects any changes you make to charts on your screen (including legend selection). You can try it out by enabling the beta feature BROWSER_WORD_EXPORT (in pancredit_parameters table) and we’d love to hear your feedback and incorporate feedback into our upcoming releases. At present it only supports certain chart types (highcharts) which doesn’t include tables, card charts, maps. Any unsupported will simply fall back to the existing export functionality. Example of the section from the new Word output: |
Improved Preview Of Chart Headings - When Categories Load/Refresh 🗣️ | pi#2532 | 31377 | Category Preview |
---|---|---|---|
Before charts are fully rendered, we populate the cell with the Chart Title to give an indication of what is loading in. Previously, this was just the Chart Name but now this preview will more closely resemble the panel heading that will appear when the chart fully loads. It will show some of the icons that will be displayed as well as the title/subtitle appropriate for that chart. Replacement cannot be done at this stage meaning any form of replacement within the title/subtitle will result in the preview not including that element. Translations, however, are available so you can expect any translated values to be represented in the accurately. |
New User/Role Permission (For Admin Users) - To Access The API Configuration Tab 🗣️ | pi#2524 | 24595 & 31607 | User/Role Permissions |
---|---|---|---|
A new User/Role permission has been added at the Admin level, to determine whether or not the User is allowed to access/edit the configuration within the APIs tab. The new “edit apis” permission defaults to ‘True’ for Admin users at the User Permission level and ‘False’ for Roles. |
JDBC Driver - Class Isolation 🗣️
pi#2489
31482
Drivers
JDBC drivers are now loaded into a separate class-loader from system libraries, significantly improving the chances of untested drivers being compatible. As part of this, the Apache Drill driver has been included in the default distribution since this was a driver which had such an issue previously.
CHANGES
Exporting - Enhanced Single Chart PDF & PowerPoint (Performance Improvements) - REMOVED FROM BETA
pi#2600
-
Exporting - PDF & PowerPoint
CHANGES
Exporting - Enhanced Single Chart PDF & PowerPoint (Performance Improvements) - REMOVED FROM BETA | pi#2600 | - | Exporting - PDF & PowerPoint |
---|---|---|---|
Our greatly enhanced, rebuilt pdf and ppt exports for single charts have been enabled by default in this release. If you experience any issue please let us know, for a limited time you can switch back to the old mode by using the beta flag OLD_PDF_EXPORT and OLD_PPT_EXPORT. |
Exporting - Enhanced Single Chart PDF & PowerPoint - A Drill Down Chart | pi#2599 | - | Exporting - PDF & PowerPoint |
---|---|---|---|
Resolved a Known Issue with exporting drilldown-level charts, when using the enhanced Single Chart PDF & PowerPoint export functionality - a Known Issue with the August 24 release. When a user exports a chart at a drilldown level, the export will now correctly capture the chart from the appropriate hierarchy level, provided the chart type is supported. If the chart type is unsupported, the export will automatically revert to using the renderer service. |
Editable Tables - Improved Performance When Sending All Data 🗣️ | pi#2545 | 32019 | Editable Tables |
---|---|---|---|
By removing some additional variable replacement that wasn’t useful we’ve improved the performance of editable tables. Especially when sending large volumes of data. |
Editable Tables - Page Resize & Other Changes Causing Duplicate Data To Be Sent 🗣️ | pi#2550 | 32019 | Editable Tables |
---|---|---|---|
Resolved an issue where page actions such as a resize could result in duplicate rows of table data could be sent to the editable tables endpoint. |
Reports - All Charts Load In Parallel - Improved Performance 🗣️ | pi#2539 | 31184 | Reports - Performance |
---|---|---|---|
Improved report performance by allowing each chart within a report to load in parallel. This does not affect how the charts are rendered, only how they are returned from the back end. Report performance logs have been updated to display the loading times for each individual chart, as well as an overall loading time, which will now be lower due to the requests for individual charts being asynchronous. |
Schedules - Improvement Reliability When Scheduling Tables (Excel Format) 🗣️ | scheduler#88 | 31192 & 31778 | Scheduler & Exports |
---|---|---|---|
Enhanced the reliability of scheduling tables in Excel format by implementing additional checks to ensure the file's integrity before returning it to the scheduler, along with improvements in naming conventions, increased logging, and better error handling. |
Analytics - Unable To Save Analytics Model | pirana#52 | 31872 | Analytics |
---|---|---|---|
Despite having the necessary permissions, Admin Users (other than Super Admin) were unable to save Analytics models to the Data Connection, due to a CORS setting issue. |
Analytics - Encoded Values Appearing On Analytics Charts | pi#2536 | 31871 | Analytics |
---|---|---|---|
Comparators like ‘>=' and special characters like '&’ would appear HTML encoded on Analytics Charts. |
Password Reset - Invalid Details Led To Exception | pi#2634 | - | Password Reset |
---|---|---|---|
Using the password reset function with invalid details previously lead to an exception being displayed. |
Organisation - Incorrect Organisation Name Showing In The Configuration Panel | pi#2648 | 33177 | Organisations |
---|---|---|---|
Within the Organisations Configuration Panel dropdown list, users will now only see Organisation values where they are either an admin or have 'schedule' permission for those Organisations. If they are an admin, all content will be displayed relating to those Organisations, otherwise an error will be returned that informs the user that they do not have access. |
Security - Resolved some issues with CORS settings | pi#2615 | - | Security | ||
---|---|---|---|---|---|
Resolved an issue where the CORS settings were not being applied correctly.
|
Performance - Several Performance Improvements 🗣️ | pi#2666 & pi#2625 | - | Performance |
---|---|---|---|
A number of areas of performance improvement were identified and implemented |
Translations - Several Improvements To Enable More Translations 🗣️ | - | 32769, 32816, 32821, 31377 | Multi Lingual |
---|---|---|---|
A number of additional areas where translations occur in the dashboard have now been added. |
SCHEMA CHANGES
Description | GitLab | ZenDesk | Column |
---|---|---|---|
Column ‘edit apis’
| pi#2524 | 24595 & 31607 | User Roles / Permissions |
MINIMUM SERVER REQUIREMENTS
The following data was created by load testing a pi server running on AWS Fargate. Load was created by automatically logging users in / out of the server. Additionally manual testing was performed during the load to observe actual performance within the system.
Explanation:
This testing was performed on the September 2024 Dashboard version. We had noticed a degradation in performance in this version particularly in the June, May and August releases.
This is not attributed to any one particular change, but rather is down to several different events / changes ;
Updates to frameworks / libraries
Java
Tomcat
Background
During our internal pre-release testing of this September 2024 dashboard release, we observed some notable performance degradation within the dashboard. This coupled with a handful of similiar issues reported in releases since our May 24 release, triggered a review of our minimum system requirements - to ensure these were set sufficiently, and to understand the cause.
Between the May 24 and August 24 releases, we have introduced several changes that have individually contributed to the overall degradation we observed, however no one specific change was the cause;
Changes to frameworks / libraries
Java
Tomcat
Grails
Angular
Removal of some server level caching
- to make provision for horizontally scaling deployments
Redis for maintaining session across application nodes
Multi-lingual additions
New
Category Object (filter) features
Organisations
Linked Data
Connections
Custom
For previous releases, the recommendation for the sizing of a single application node for nominal load was given as 1 vCPU and 2GB RAM
We have tested additional combinations and our explanations, results and conclusions are below.
There have also been twoFields logic
Two performance related code changes
Better handling of encrypted variables
Application Optimisation to have also been introduced in this September 24 release
Improved handling of Encrypted Variables
Application Optimisation - to reduce requests when refreshing the page
Analysis & Testing
The chart below logs the output produced by the dashboard (MIS_AUDITS) and shows the number of chart requests served by the dashboard per minute. We use other metrics – but this served rate has proven to be highly correlated to performance and the user experience when interacting with the system. The vast majority of requests are serving charts. We tested the ability to perform configuration actions at the same time.
The green at the same time.
Note |
---|
Load Testing was performed on pi server running on AWS Fargate. Load was created by automatically logging users in / out of the server, in addition to manual tests being conducted during the load, to observe actual performance. The repository we tested against included volumized users, categories, organisations and data connections. We also included (and used) Secured Variables too. |
The Green band shows the high load areas we observe on customers dashboards that run on our Pi SaaS instances.
The pink Pink line shows the point of instability we noticed on the during our internal pre-release September version, testing for the September 24 release - that caused nodes to become unstablebe impacted. We were able to observe this consistently, and this was before the code fixes were applied and we made subsequent code changes to address the issues, and were based on a node with 1vCPU and 2GB RAM (these being our previous recommended settings for a single application node).
We used a repository with volumised users, categories, organisations and data connections. The repository also included (and used) secured variables.
On these runs we altered the number of vCPU’s, RAM and maxThreads environment variable.
Matrix
To observe the impact of changing resources, we performed multiple test runs, making amendments to
vCPU’s
RAM
maxThreads (environment variable)
The table below details the finding we recorded for each run and configuration change.
Run | vCPU | RAM (GB) | Max Threads | Observed |
1 | 2 | 4 | 20 | System hit a limit at 170 charts served per minute. The server remained stable at this point, and when load was reduced continued to operate efficiently. |
2 | 2 | 4 | 30 | The system performed well and was able to perform to ~350 chart requests. The server remained stable |
3 | 2 | 4 | 40 | The system performed well and was able to perform to ~400 chart requests. The server remained stable |
4 |
|
| 50 | The system was still able to perform to ~350 chart requests per minute, but the observed performance was significantly reduced. |
5 | 1 | 4 | 40 | The system was significantly slower even whilst no load was provided. We struggled to load the system beyond ~120 charts per minute. The server did remain stable. |
6 | 2 | 8 | 40 | The system was able to perform ~470 charts requests per minute, was stable and user experience was good even while loaded. It was observed that the additional 4GB of RAM was not used by the system. |
7 | 2 | 4 | 40 | Following test 6, we re-performed test 5 to confirm that the additional 4GB of RAM was unnecessary, and the experience on this test 5 and 6 was comparable. |
8 | 2 load balanced * 2vCPU | 4GB | 40 | Where able to show that the resources where effectively run over load balanced nodes, and achieved ~1,000 charts per minute, and the solution was very stable even under this load. |
9 | As Above |
|
| Confirmed that the system could be loaded and unloaded without effect. |
Results & Recommendations
Our current recommendation is that a pi Node should be as follows.
Minimum Requirements Based on our findings, we recommend that the following resource and configuration changes are applied to ensure optimal performance and stability of application nodes.
Resource Requirement
2vCPU
4GB RAM
Configuration Changes Recommended SettingsWe recommend - the following configuration change.
Tomcat MaxThreads should be set to the same value as the repository MaxConnections (our recommended value is 40)
PI_TOMCAT_MAX_THREADS = 40
PI_TOMCAT_MAX_CONNECTIONS = 40
Also, with the increase in memory to 4GB RAM, you should make sure 3GB is allocated to tomcatthe Tomcat, leaving 1GB for the operating system.
PI_TOMCAT_MAX_MEMORY = 3072