Open

We are pleased to present our latest offering of pi - our July 2022 release! This page helps summarise and familiarise you with the changes we have made, and is supported (where applicable) with useful videos to help explain the changes we have delivered.

Here’s what we’ll cover;

Patch Release

A critical level remote code execution vulnerability was reported in Grails, an upgraded version of Grails has been added to this patch release to address this. Please refer to this page for more information.

Previously, if you had set up a regular or crosstab table as a parent chart, then re-ordered the rows by clicking on the table header, then clicked on an item within the table to filter on, it would filter on the incorrect item. It would filter on what was at that same position in the table before it had been re-ordered by clicking the header.

Now, it filters on the correct item regardless of whether or not you have re-ordered it.

When a user that has the following options enabled in their user account:

• Is User Locked

• Captcha At Login

• Password Expired

and the user tries to access the dashboard and enters a new password at the change login screen the user can log in to the dashboard.

Added

Core framework Grails updated to version 3.

Rate limiting functionality has been added to data connections, this will limit the number of attempts you can make for testing and saving a connection for security reasons.

You can make up to a total of 20 attempts for saving and testing a data connection within 600 seconds (10 minutes) before an error is displayed and you are prevented from making any more until 600 seconds (10 minutes) have passed since making the first stored attempt.

Rate limiting for testing (hitting the lightbulb icon) and saving a data connection are stored separately, this means that you can make 20 attempts within 10 minutes for EACH piece of functionality.

Open

Users will now have the option to enable HTTP Strict Transport Security (HSTS) when serving the dashboard over HTTPS. This is recommended, and provides an extra layer of security by converting all attempts to access the site using HTTP to HTTPS. HSTS can only be enabled when serving the dashboard over HTTPS and can be configured through the configuration tool using the options below:

• Enabled - this switches on HSTS.

• Include Subdomains - Applies HSTS to all subdomains of your site.

• Preload - By enabling preload and adding your site to Google’s HSTS preload service, browsers will never connect to your domain using an insecure connection.

  • You should only enable preload if you’re confident your entire site and all of its subdomains can be served over HTTPS. Once on the HSTS preload list, it can take several months to remove your site if needed.

• HSTS Maximum Age - The time, in seconds, that the browser should remember that a site is only to be accessed using HTTPS. This is set to one year as default.

HSTS can also be configured with the following environment variables:

• PI_TOMCAT_USE_HSTS - Boolean field. Apply HSTS to the dashboard.

• PI_TOMCAT_HSTS_INCLUDE_SUBDOMAINS - Boolean field. Applies HSTS to all subdomains of your site.

• PI_TOMCAT_HSTS_PRELOAD - Boolean field. Enables HSTS preload header, meaning browsers will never connect to your domain using an insecure connection (see above warning).

• PI_TOMCAT_HSTS_MAX_AGE - The amount of time in seconds that the browser should remember that the site is only to be accessed using HTTPS. Default setting of 31536000.

Further to being able to configure HSTS, the dashboard now has Permissions-Policy and Feature-Policy security headers as standard when serving the dashboard over HTTP or HTTPS. This includes:

After 3 failed login attempts, rather than locking a user account, the user is now shown a captcha image to fill on login. Accounts will now be locked after 10 failed attempts.

The number of failed attempts it takes to show the captcha and to lock the account are configurable in Dashboard Configuration>Settings:

Additionally, an admin user can force (or remove) the captcha state for a user (much like they could toggle the lock state of a user) in Dashboard Configuration>Users:

There is now a health check endpoint available as part of the dashboard to verify whether the dashboard is running. This provides a simple json response simply showing the status of the dashboard (e.g. {"status": "UP"}). Users can navigate to the endpoint by simply adding "/health" to the end of the relevant url (e.g. https://localhost:9090/pi/health).

A new API endpoint has been added that allows you to invalidate your token.

Make a post request to this endpoint, with your token in the authorisation, and it will be invalidated:

http://host:port/pi/api/v2/tokens/invalidate

We’ve added the ability to suppress password login for users that are using an external login service. This can be configured in Dashboard Configuration>Users.

The new checkbox ‘Allow Password Login’ is ticked by default, when un-ticked it turns off the user’s ability to log in with their password and also prevents them from changing any of their details within the account panel as a password is required to do this.

When this option is unticked, the password field and password expires checkbox will not be visible. The password will be saved as ‘null’ in the database.

You will be unable to save a user with both password and external login disabled.

Also, when checking password login, after having had it previously unchecked for that user, you will be unable to save unless you add in a new password, as it would have previously been saved as ’null' in the database.

Changed

A maximum limit of 200 characters has been added for the file names of chart downloads. If the chart name, or title, added in chart editor exceeds 200 characters the file name will be truncated which will allow the file to download successfully.

Previously, after hovering on a sparkline on either a:

• Card chart

• Speedometer

• Lightbulb

• Traffic light

• Power ring

the value of the last position that you hovered on on the sparkline would be retained when the mouse was moved away from the sparkline.

Now, when the mouse is moved the sparkline will reset to the original value (the value it was before a user hovered over it).

Example:

Hover on a value on the sparkline, the chart reflects the value.

Move mouse cursor away, the chart resets to the original value it showed before you hovered over the sparkline - in this example, 180.45.

Previously, if you defined a colour in the chart editor for a table and selected ‘Apply Colour To Whole Row’, the colour would only show on the cells that didn’t have a colour already defined for them in the themes. By default, our total column has a colour defined in themes, so it would often appear as if the ‘Apply Colour To Whole Row’ checkbox wasn’t being displayed on the total column.

Now, when you define a colour in chart editor and select ‘Apply To Whole Row’ it will override whatever is defined in themes and therefore, the total column will also be coloured.

A new user, ‘pi_repos_user’, has been added to replace a ‘root’ user for the Dashboard Repository data connection. This new user has ‘read' privileges only which limits unnecessary access to the information schema tables. The ‘root’ user will be replaced by the 'pi_repos_user’ on a new dashboard installation or dashboard upgrade to the July release.

A password for this new user will be stored at %DASHBOARD_INSTALLATION_PATH%/Dashboard/db/pi_repos_user.log.

When selecting a legend item on a multi-series parent chart, there will no longer be a change triggered in the url. In the previous implementation this would have filtered all values on the X Axis, only allowing for deselection of items on the axis. In the new implementation this allows for filtering on the parent chart itself, as well as the selecting of individual items on the X Axis after selecting by legend.

When clicking on the legend in a multi-series parent chart, the parent chart should behave the same as if it wasn't a parent chart, and no changes should occur on any other charts in the category.

Multi-series parent chart - nothing selected

In the following example, the bar chart has been filtered by the item selected in the legend and the table on the right remains unchanged.

Multi-series parent chart - selected by legend

If you select a value on the X Axis before selecting by legend, the parent chart will filter as expected and only show the data of the relevant selected item in the parent chart and any other charts on the same category.

Multi-series parent chart - selected by x-axis and legend

Puppeteer library has been upgraded to version 14.1.2. This fixes the problem with the chromium browser throwing status_stack_buffer_overrun error in certain environments.

Fixed

Users will now be able to get multiple tokens via api in parallel for a user, whose password does not expire, on a clean fresh db.

When a user double clicked the ‘Save’ icon in quick succession on any screen in Dashboard Configuration that contained a ‘Save’ icon, an error message was being displayed. With effect from this release of the dashboard, this error message will no longer be displayed.

When users tried to change a bar chart to a different type of chart, in Chart Editor, the chart types were overlapping on the Chart screen after the new chart type had been selected.

In the following example, we are changing a Bar chart to an Area chart. After searching for the Area chart and clicking on it, the two chart types are overlapping.

From this release, all chart types will now display correctly in Chart Editor.

Users were not able to add a new or amend an existing data connection when using Dashboard Configuration>Data Connections. Releases prior to July 2022 were not affected.

Drill Down were not working correctly when created on charts in Categories that contained a Date Range Category Object.

User restrictions that were created in the API were not present in the dashboard.

The classpath value was not saved when creating or modifying a free-format jdbc connection.

Users were unable to change the colours on the Analytic chart type.

Charts that were set as anonymous were not being displayed without users having to log in to the dashboard.

Drill down levels were not working on the Analytic chart type.

Users were unable to override a data connection in dashboards that only contained one data connection.

When un-ticking the ‘Filter’ checkbox in Dashboard Configuration> Categories, the category object was not being replaced. Category objects are now being replaced correctly.

Merged tables were missing content when exported as PowerPoint or Word single downloads. Now, the Word & PowerPoint downloads will display all rows of the merged table. In addition, once the file is opened, the rows of the table will also be editable instead of being sent as an image.

Migrations were not running when the ports had been changed.

There were some inconsistencies with how values were displayed on charts due to the way decimals were rounded. The total was previously calculated by adding all the individual values together and then rounding the total.

Each individual point will now be rounded and then added together to produce the total figure.

This change affects:

• Data visualisations

• Tooltips

• Data labels

• Dynamic targets

For the following:

• Individual points

• Total points

• Cumulation

• Decumulation

• Percentages

in all chart types where relevant.

In addition to the above, visualisations for the scatter and bubble charts & dynamic targets will now also display rounded values.

When a chart has been deleted from a Category, using Category Access, the error message displayed when users view this Category will now say ‘This chart has been deleted’.

In Chart Editor, any attribute that is toggled on or off will contribute to the count shown beneath the Attributes tab. In the following screenshot, we can see that 2 attributes have been selected.

If the category object panel is pinned, and no Category Objects are being used on the category the category objects panel will not be displayed. For example, if you were using Category Objects and pinned the category objects panel and then removed the Category Objects the category objects panel would be automatically unpinned and hidden from view on that Category.

When targets have been added to a chart, a number representing the number of targets will be visible beneath the Target tab in Chart Editor.

If the chart type is then changed to a chart type that does not support Targets, the target settings will be removed.

This will also apply to Filters and Sorting if the chart type is changed to a chart type that does not support these features.

Cells that contain Lightbulb charts, or any chart type that includes a sparkline for example, a Speedometer, can now be made smaller vertically by dragging the cell border.

The ‘Default Colour’ field on the Colours tab, when editing data tables in the Edit Chart Screen, has been removed. This is because data tables don’t support default colour definition.

Undesirable border on the dashboard has been removed so that when embedding the dashboard using an iframe a left and a right border is not displayed.

In Chart Editor for Combined Charts, the Attributes tab will now correctly show the chosen Chart Types.

In the following example, we can see that Area has been selected for the first Y Axis and Bar for the second Y Axis. This information is correctly reflected in the Matrix grid beneath the chosen chart types.

In very rare cases, if the dashboard data connection details were not stored in the expected record, the system tried to update the connection password when the dashboard was started. This has been changed, so the system only updates the password to be in sync with the one stored in the file if we know for sure the target connection is the dashboard connection.

An error was occurring, and the file was not downloaded, when a table was exported to Excel if cells in the table contained a lot of text. With effect from this release of the dashboard, this will no longer happen.

Empty charts were causing scheduler to display an error relating to incorrect user permissions. In this release, if the chart is empty and the ‘Data’ tab is not ticked, an empty chart will be scheduled to email.

An issue with the card chart and analytics chart types being scheduled has also been resolved. If a card chart or analytics chart type is being scheduled, it will now send out an email with no errors being displayed.

When a Category Object with a default value has been added to a Category and a new chart is created on that Category (in an empty cell) the Category Object will be visible on the Filters screen in Chart Editor when the new chart is being created.

In this example, the Category contains a Category Object with a default value of 2022.

When we add a new chart to this Category, the Filters screen, in Chart Editor, correctly shows the default value for the Category Object.

Using the CTRL click method, users will be able to select multiple items from a Multi Picker category object within a 1.5 second time frame before the search box closes and the query is executed.

When object referencing (#~ObjectName~#) has been used in a Table WHERE clause, charts with such setup will now be displayed correctly.

Previously, there was an issue where apostrophes/single quotes were not being escaped within category objects referenced within certain places needed for the SQL. This included:

• Table SELECT statement

• Table WHERE clause

• Free hand SQL on the chart editor

Additionally, whilst addressing this issue we noticed that category objects were not being replaced at all within the order by statement in the data objects and have also made this change.

When users clicked on the ‘Reset to the default view of the category’ icon on a Category that contained a Parent and Child chart and the browser console was open, the default view was reset but an error message was being displayed in the browser console. This will no longer happen with effect from this release of the dashboard.

When the system contained a category with a category object that was set to drop list and the same category was imported containing a multi-select list, the target system was not being updated with the correct category object settings i.e. multi-select.

All special characters within Excel spreadsheet headers and table names are now converted to either an underscore or a blank space, so they can now be successfully saved to the database.

You can find more information here.

When users clicked the ‘Schedule this chart’ icon in Chart Tools, the Schedule screen was not displaying a list of users to select from. With effect from this release of the dashboard, the list of users will be displayed as expected.

There was a problem when using category filters with index fields to replace text on a filter, the behaviour was changed in the previous release. This is to revert the change so that some very specific scenarios work the same as before.

If a chart has an error, users will still be able to edit the chart by clicking the ‘Edit chart’ icon in the Chart Tools menu.

A new CSS class has been added called pi-style__category-objects_panel-content. This allows users to add gaps to the whole Category Object panel. Without this addition, adding gaps to pi-style__category-objects-top-panel resulted in unnecessary gaps when there were no category objects on the screen.

The new CSS class can be found in Dashboard Configuration>Themes>Style>Dashboard.

When a user changes their own theme by clicking their name in the ‘More Options’ section at the top-right of the dashboard

they will need to specify their password in order to save the changes.

The ‘Can Change Own Password' and 'Can Change Own Theme' permissions are now linked to the 'Can Access Account' permission. Users need to be able to access the Account screen, accessed by selecting the user name under 'More Options’ at the top-right of the dashboard, to change their password and theme.

If you tick either the ‘Can Change Own Password’ or ‘Can Change Own Theme’ permissions in Dashboard Configuration>Users, the ‘Can Access Account’ permission will be selected by default.

Similarly, if you untick the ‘Can Access Account’ permission, ‘Can Change Own Password’ and ‘Can Change Own Theme’ will also be unticked.

When background opacity has been set on .pi-style__login-screen__background-image the opacity will only apply to the image and not the entire logon screen.

Undesirable CSS definition has been removed on .pi-style__page-top-nav__logo so that !important is no longer needed in order to define line-height and height.

Images and text will now be retained on Card Charts that do not contain Dimensions and Measures.

Previously when a Card Chart didn’t have a dimension or measure, but contained a drill down level, the image was removed after saving the chart.

In some instances, import of data connections would fail due to multiple objects with the same identifier being present in memory. This has been resolved in this release of the dashboard.

When the height of a category was set to anything other than the default setting, for example 1.5 or 2, users who did not have the ‘Can Modify Layouts’ permission were unable to scroll in the category.

When a report was created that included a merged table, only part of the data in the report was sent when the report was scheduled.

Importing a data connection from an earlier release of the dashboard was failing due to a constraint error.

Embedding was not working when updating from an earlier version of the dashboard to the current dashboard release.

When importing a data connection that has different connection details to the system you are importing it into, unticking this box will ensure that the connection details are not overwritten.

Reports that contain a header or footer were not displaying correctly when exported.

An error message was being displayed when a clean install was done with non-default ports.

The dashboard session timeout (set in Dashboard Configuration>Settings>General Settings was not being honoured in Grails 3.

Data connection introspection (any test button) was not working correctly.

Windows authentication support was disabled after significant upgrades recently required the functionality to be reworked. The functionality is now restored for customers wishing to use it.

When using the API to import a data connection to override an existing connection, in order to include connection details, an additional flag is needed for this version of the dashboard.

So previously the request content was:

This version needs the isConnectionSelected flag

Note:

• This issue only affects the API. There is no problem if you use the UI for connection importing

• This issue only affects importing a connection to override one with the same name in the system. If the system doesn't have such a connection, this flag is not needed

• This flag only affects data connection details. It doesn't affect object, charts, categories and so on, which are working exactly the same as in previous versions of the dashboard

• We are fixing this problem, so the next version of the dashboard will not need this additional "isConnectionSelected" flag

The Scheduler was displaying an error in the UI, which was related to user permissions when a merged table or merged chart was being scheduled. The scheduler will now send out the merged tables and charts correctly with no errors present in the dashboard.

When using the default theme, emailed reports were not showing the lines or bars in charts. This was only happening when using the default theme - all other themes were working as expected.