Disclaimer

This document is only collection of author’s notes, experiences and point of views. It is not, in any meaning, either complete description of the topic nor official RTB documentation. It may be inaccurate, incomplete, obsolete, misleading or completely wrong. It may even cause loss of data or damage system integrity. It may not comply with company codex, values, presentation style or economic interests or may reveal company secrets.

As such, it is absolutely prohibited to distribute this document outside of RTB & Co. GmbH. Such an action may result into legal acts against both the sender and the company. It is only intended, after review of technical and presentational correctness and accuracy, to be used as an information source for official documentation.

Always contact documentation department for information about current presentation style and allowed formats before creating customer’s documentation.

Environment

Component Version

Java

17

PostgreSQL

14

Tomcat

9

22.0.0

There are some compatibility breaks. Please read following section carefully.

Database issues

There are several issues connected with data security and usability.

Super user access

So far it was possible to run PDM.control only with postgres user. It is not such a big problem when PDM.control is the only application which uses PostgreSQL database. However, this is far from ideal in multi-application environment. Such a setup does not provide enough security and isolation. An error in the application may affect the other databases because postgres user is database superuser and therefore it can access every database within the cluster. To avoid this, every application should run under its own user with permission to work with just the database which it needs.

PDM.control 22 does not require to use postgres user any more. Next steps describe migration process.

Public schema and search path

Each PostgreSQL has public schema which is by default accessible to all users. It means that anybody can read / write to it no matter which role he has. This schema is used by PDM.control. Public access has to be denied. Additionally there PDM.control’s tables will be moved to new schema control to logically group tables and allow future extendibility.

Database creation

I is not possible to make in installations of PDM.control because scripts which create database fail. It is caused by incompatibly of new version of migration library. Unfortunately it is not possible to simply fix migration scripts. They must stay constant for whole application life time. Otherwise would PDM.control not start without manual change in database.

Migration

Following steps describes how to migrate existing database of to be compatible with PDM.control 22.

Pre-requisite

Install latest version (21.8) first and let database upgrade process to complete. Readiness of the database for migration will be verified at the beginning of migration script.

Migration

Stop PDM.control.

Large objects

First release unused large objects by following command

vacuumlo -v -h localhost -p 5432 -U postgres pdm_parking

Start migration

To avoid changes in migration script below is recommended to temporarily rename database to pdm_parking. PDM.control user name `pdm_control and password from script can be also temporarily accepted. Both can be changed later.

Connect database pdm_parking as superuser (postgres).

Begin new transaction be be able to recover in case of error.

BEGIN;

Run following script (as postgres user) to migrate the database.

-- Test if database is ready for migration. If not an exception will be raised, which stops script execution.
DO $$
DECLARE
	last_change_set varchar;
BEGIN
	SELECT id INTO last_change_set FROM public.database_change_log ORDER BY id DESC LIMIT 1;
	IF last_change_set = '220204T1222' THEN
		RAISE NOTICE  'Database is ready to be migrated to version 22.';
	ELSE
		RAISE EXCEPTION  'DATABASE IS NOT READY. PLEASE INSTALL PDM.CONTROL 21.8 FIRST.';
	END IF;
END;
$$;

-- Create user pdm_control
CREATE USER pdm_control PASSWORD 'bYue[IOZ3"oc%r5T*BCT';
-- Let user pdm_control own the database
ALTER DATABASE "pdm_parking" OWNER TO pdm_control;
-- Move ownership of public schema.
ALTER SCHEMA public OWNER TO pdm_control;

-- Create new schema for PDM.control.
CREATE SCHEMA control AUTHORIZATION pdm_control;

-- Drop liquibase change logs
DROP TABLE database_change_log;
DROP TABLE database_change_log_lock;


-- Alter ownership of all objects to uses pdm_control;
-- It needs to be done in multiple  steps because custom types are not part of standard SQL.

DO $$
DECLARE
	-- Configuration
	v_schema varchar := 'public';
	v_new_schema varchar:= 'control';
	v_new_owner varchar := 'pdm_control';
	-- Internal variables
	r record;
	cmd_owner varchar;
	cmd_schema varchar;
BEGIN
	-- Move only tables (except liquibase and users, they will stay in public schema). Owner will be changed in new schema.
	RAISE INFO 'Moving tables ...';
	FOR r IN
		-- Table owner
		SELECT 'ALTER TABLE "' || table_schema || '"."' || table_name || '" OWNER TO ' || v_new_owner || ';' AS a FROM information_schema.tables WHERE table_schema = v_schema
		union all
		-- Table schema (leave liquibase tables in public schema)
		SELECT 'ALTER TABLE "' || table_schema || '"."' || table_name || '" SET SCHEMA ' || v_new_schema || ';' AS a FROM information_schema.tables WHERE table_schema = v_schema
	LOOP
		RAISE NOTICE '%', r.a;
		EXECUTE r.a;
	END LOOP;

	-- Rest of operation must be done in new loop because tables now belong to new schema.
	RAISE INFO 'Moving functions, views, sequences ...';
	FOR r IN
		-- Sequence owner
		SELECT 'ALTER TABLE "' || sequence_schema || '"."' || sequence_name || '" OWNER TO ' || v_new_owner || ';' AS a FROM information_schema.sequences WHERE sequence_schema = v_schema
		union all
		-- Sequence schema
		SELECT 'ALTER TABLE "' || sequence_schema || '"."' || sequence_name || '" SET SCHEMA ' || v_new_schema || ';' AS a FROM information_schema.sequences WHERE sequence_schema = v_schema
		union all
		-- View owner
		SELECT 'ALTER TABLE "' || table_schema || '"."' || table_name || '" OWNER TO ' || v_new_owner || ';' AS a FROM information_schema.views WHERE table_schema = v_schema
		union all
		-- View schema
		SELECT 'ALTER VIEW "' || table_schema || '"."' || table_name || '" SET SCHEMA ' || v_new_schema || ';' AS a FROM information_schema.views WHERE table_schema = v_schema
		union all
		-- Function owner
		SELECT 'ALTER FUNCTION "'||nsp.nspname||'"."'||p.proname||'"('||pg_get_function_identity_arguments(p.oid)||') OWNER TO ' || v_new_owner || ';' AS a FROM pg_proc p JOIN pg_namespace nsp ON p.pronamespace = nsp.oid WHERE nsp.nspname = v_schema
		union all
		-- Function schema
		SELECT 'ALTER FUNCTION "'||nsp.nspname||'"."'||p.proname||'"('||pg_get_function_identity_arguments(p.oid)||') SET SCHEMA ' || v_new_schema || ';' AS a FROM pg_proc p JOIN pg_namespace nsp ON p.pronamespace = nsp.oid WHERE nsp.nspname = v_schema
	LOOP
		RAISE NOTICE '%', r.a;
		EXECUTE r.a;
	END LOOP;

	-- Types needs to be moved by extra command. They are not part of standard SQL therefore to command is more complex. Source: https://stackoverflow.com/a/3703727
	RAISE INFO 'Moving types ...';
	FOR r IN
		SELECT n.nspname as schema, t.typname as type
		FROM pg_type t
		LEFT JOIN  pg_catalog.pg_namespace n ON n.oid = t.typnamespace
		WHERE (t.typrelid = 0 OR (SELECT c.relkind = 'c' FROM pg_catalog.pg_class c WHERE c.oid = t.typrelid))
		AND NOT EXISTS(SELECT 1 FROM pg_catalog.pg_type el WHERE el.oid = t.typelem AND el.typarray = t.oid)
		AND n.nspname = v_schema
	LOOP
		cmd_owner:= format('ALTER TYPE "%s"."%s" OWNER TO %s', r.schema, r.type, v_new_owner);
		cmd_schema:= format('ALTER TYPE "%s"."%s" SET SCHEMA %s', r.schema, r.type, v_new_schema);
		raise notice '%', cmd_owner;
		EXECUTE cmd_owner;
		raise notice '%', cmd_schema;
		EXECUTE cmd_schema;
	END LOOP;
END$$;

-- Re-declare function 'add_default_message_config' in new schema 'control'
DROP FUNCTION IF EXISTS control.add_default_message_config(integer, integer, control.status_message_alert, control.status_message_impact, boolean, boolean);
CREATE OR REPLACE FUNCTION control.add_default_message_config(
	in_msg_nr integer,
	in_solve_nr integer,
	in_alert_type control.status_message_alert,
	in_msg_impact control.status_message_impact,
	in_manual boolean DEFAULT false,
	in_important boolean DEFAULT false)
		RETURNS void
		LANGUAGE 'plpgsql'
		COST 100
		VOLATILE PARALLEL UNSAFE
AS $BODY$
DECLARE
BEGIN
	-- Sanity checks
	IF in_msg_impact = 'NEGATIVE' AND in_solve_nr IS NULL THEN RAISE EXCEPTION 'Negative message has to define solve number.'; END IF;
	IF in_msg_impact IN ('POSITIVE', 'TRANSIENT') AND in_solve_nr IS NOT NULL THEN RAISE EXCEPTION 'Positive and transient message must not define solve number.'; END IF;
	IF in_manual IS NULL THEN RAISE EXCEPTION 'Manual mode must not be null'; END IF;
	IF in_important IS NULL THEN RAISE EXCEPTION 'Important mode must not be null'; END IF;
	IF in_msg_impact = 'POSITIVE' AND in_alert_type != 'NONE' IS NULL THEN RAISE EXCEPTION 'Alert type for positive messages has to be NONE'; END IF;

	-- Test solve message properties
	IF in_solve_nr IS NOT NULL
	THEN
		-- Test if referenced message exists
		IF (SELECT COUNT(*) FROM control.message_config_default WHERE mcd_number = in_solve_nr) = 0 THEN RAISE EXCEPTION 'Referenced positive message % does not exits.', in_solve_nr; END IF;
		-- Test if referenced message is really positive
		IF (SELECT mcd_impact FROM control.message_config_default WHERE mcd_number = in_solve_nr) != 'POSITIVE' THEN RAISE EXCEPTION 'Referenced positive message % is not positive.', in_solve_nr; END IF;
	END IF;

	INSERT INTO control.message_config_default(mcd_number, mcd_solve, mcd_alert, mcd_impact, mcd_manual, mcd_important)
	VALUES (in_msg_nr, in_solve_nr, in_alert_type, in_msg_impact, in_manual, in_important)
	ON CONFLICT (mcd_number)
	DO UPDATE SET mcd_solve = in_solve_nr, mcd_alert = in_alert_type, mcd_impact = in_msg_impact, mcd_manual = in_manual, mcd_important = in_important;
END;
$BODY$;

ALTER FUNCTION control.add_default_message_config(integer, integer, control.status_message_alert, control.status_message_impact, boolean, boolean) OWNER TO pdm_control;

-- Re-declare comment on function function extendible_permits in new schema 'control'
COMMENT ON FUNCTION control.extendible_permits(integer, character varying, integer) IS 'List all extendible permits having provided lpn OR psn. Use null for not provided value. Both values can be provided simultaneously.';

-- Mark change log for liquibase
CREATE TABLE IF NOT EXISTS public.database_change_log
(
	id character varying NOT NULL,
	author character varying NOT NULL,
	filename character varying NOT NULL,
	dateexecuted timestamp without time zone NOT NULL,
	orderexecuted integer NOT NULL,
	exectype character varying NOT NULL,
	md5sum character varying,
	description character varying,
	comments character varying,
	tag character varying,
	liquibase character varying,
	contexts character varying,
	labels character varying,
	deployment_id character varying
);
ALTER TABLE public.database_change_log OWNER to pdm_control;
ALTER TABLE public.database_change_log ADD CONSTRAINT pk_db_changelog PRIMARY KEY (id, author);

INSERT INTO public.database_change_log (id, author, filename, dateexecuted, orderexecuted, exectype, md5sum, description, comments, tag, liquibase, contexts, labels, deployment_id)
VALUES ('220216T0000', 'ales.holec', 'db/022/220216_db_init.sql', '2022-02-16 15:07:51.844589', 1, 'EXECUTED', '8:aa1cff7923b36796cec821452f59b568', 'sql', '', NULL, '4.5.0', NULL, NULL, '5020469796');

Next change ownership of large object issuing following command

DO $$
DECLARE
	oid_to_update oid;
	cmd_owner varchar;
BEGIN
	RAISE INFO 'Updating large objects ...';
	FOR oid_to_update IN
		SELECT DISTINCT sws_data FROM control.software_item
	LOOP
		cmd_owner:= format('ALTER LARGE OBJECT %s OWNER TO pdm_control', oid_to_update);
		EXECUTE cmd_owner;
	END LOOP;
END;
$$;

This may throw error similar to this one

ERROR:  out of shared memory
HINT:  You might need to increase max_locks_per_transaction.
CONTEXT:  SQL statement "ALTER LARGE OBJECT 69059 OWNER TO pdm_control"
PL/pgSQL function inline_code_block line 13 on EXECUTE
SQL state: 53200

In this case, ROLLBACK the transaction. Then go to postgresql server configuration and increase value of variable max_locks_per_transaction to some higher value.

E.g.

max_locks_per_transaction = 512

Restart server and repeat the upgrade process. After successful migration set original value of the parameter max_locks_per_transaction back to its original value.

If the migration was successful (there are no errors in output) then the transaction can be committed. Run command

COMMIT;

If there was an error rollback transaction and call our help desk.

ROLLBACK;

If you renamed database to pdm_parking rename it back. Be sure that database is not open by any session.

ALTER DATABASE pdm_parking RENAME TO my_database;

You can also rename the user and change its password.

ALTER ROLE pdm_control PASSWORD 'xxxxxx';
ALTER ROLE pdm_control RENAME TO my_user_name;

De-fragmentation

This downtime can be also (optionally) used to physically de-fragment database. Become pdm_control user and run following commands. It may take some time based on database size.

-- Clearings
CREATE INDEX tmp_clearing_timestamp ON control.clearing USING btree (clr_date_pdm ASC NULLS LAST);
CLUSTER control.clearing USING tmp_clearing_timestamp;
DROP INDEX control.tmp_clearing_timestamp;
-- Status
CLUSTER control.logbook USING idx_logbook__date_pdm;
-- Payments
CLUSTER control.translog USING fki_translog__pdm;

Update PDM.control

Install PDM.control 22.0 Change configuration of database connection to use newly created user pdm_control with password bYue[IOZ3"oc%r5T*BCT like this.

spring:
  datasource:
	url: jdbc:postgresql://localhost:5432/pdm_parking
	username: pdm_control
	password: 'bYue[IOZ3"oc%r5T*BCT'

Start PDM.control 22.0

New database for version 22

Log in as postgres user and create user pdm_control if not exists.

CREATE USER pdm_control PASSWORD 'bYue[IOZ3"oc%r5T*BCT';

Create new database for PDM.control using postgres user.

CREATE DATABASE pdm_parking OWNER pdm_control;

Set credentials of pdm_control to PDM.control configuration.

spring:
  datasource:
	url: jdbc:postgresql://localhost:5432/pdm_parking
	username: pdm_control
	password: 'bYue[IOZ3"oc%r5T*BCT'

Start PDM.control.

Deployment

This is only tip. We did not test it. JAR file, which is available since version 20, with embedded Tomcat can be also used from Windows. For further information refer to article Spring boot as windows service or Windows Service Wrapper project page.

New features

Firmware management

Management of PDM firmware was improved. It is possible to view history of firmware updates. Results can be filtered by target or status. Also only latest updates can be displayed.

There is also possibility to download the file which was sent to PDM. Link between download plan and software was added.

PDM.control do not store original file in case of archives. Please note following limitations:

  • Configuration bundle: There may be difference in HASH, content is same as original file.

  • PDM 5 firmware: Only *.h66 file are stored in database. Files *.loc files are ignored by PDM.control. Downloaded file can be normally used for another update. However files which are not relevant for update are lost. Such a file is not equal to file got from original RTB original distribution.

File download

It is possible to download arbitrary file from PDM. Please contact our help-desk to get instructions how to use it.

Brain behind

There is a support for CU_SERVICE function provided by Brain Behind. Please contact our help-desk to get instructions how to use it.

Restricted parking place

There is a support for parking places with barrier. See requiremets and implementation. Please contact our help-desk to get instructions how to use it.

Additional changes

  • Time selector for logbooks and reports allows to select time with minute precision.

  • Pager does not display buttons for ±10 pages if there is less then 10 pages.

  • Rules of bonus "multi tariff 2" can be deleted.

  • Rules editor and QR code generator for bonus "multi tariff 2".

22.1.0

If you use integration with WES please fill integration parameters in appropriate area partner. WES configuration is not hardcoded any more. Original WES URL is: https://wes-fe-srv-01.venis.it:8443/services/5.0/public/validateParkingPass

Compulsory zones

From now on every PDM must be in Zone. If Area contains PDMs which not assigned to any Zone then new Zone will be created during PDM.control update. Un-assigned PDMs will be automatically moved to it. If Area already contains multiple Zones one of them will be marked as default.

One of Zones must always be default. Which one it is can be selected in Area properties. This Zone will be used for newly created PDMs after they report to PDM.control for the first time. If there is a plan to add new PDMs to new Zone, create the Zone first and mark is as default before PDMs are activated.

Deleting Zone was changed. It is on purpose not possible to delete Zone which contains at least one PDM or it is default. If there is a need to delete such a Zone move PDMs to some other one.

HKS

It is possible to use PDM control together with HKS partner. If you want to use HKS services please contact out help-desk or HKS directly.

Custom PDM and Area ordering

It is possible to define how PDM, Areas and Users in various lists are sorted. Sorting strategy can be set in user profile. It affects all view where list of PDMs, Areas or Users are presented. Value is stored in browser local storage. It means that it can be set on each computer individually.

Force change update state for download item

In the update plane is option to forcedly set state of each update entry. User has to have role 'ROLE_PCON_SOFTWARE_INSTALL_MANAGE' to be able to perform the action. Do not press any of those new buttons unless you are absolutely sure what you are doing.

Integration with FTT, Kladno

This feature integrates FTT solution with PDM.control. It can be enabled in partner configuration on Area tab. If it is enabled it sends every payment to FTT for further processing. See application log for errors.

Important

User zone identification

User’s identification of zone has to be set according instructions from FTT. For Kladno it is KL1KL6. Other towns may have other identifications. If user User’s identification is not provided then a warning is shown in log. Also error of failing registration can be seen.

Error handling

If there is an registration error then a log message is written. There is also possible to set and e-mail address which will notified in such a case.

CPU detection

Resources allocated by PDM.control during start are derived from number of CPU. This works well but sometimes (mainly from experimental reasons) it is necessary to set number of CPU in configuration. Now a property pcon.development.cpu-count can be used to set it. Following values are allowed:

  • < 0 - automatic detection. (default)

  • > 0 - number of CPUs to be considered.

  • = 0 - minimal resource consumption.

WebClient logging

Do not use in production permanently! It is possible to log communication of between PDM.control and partner. It is useful to debug some issues. There are two steps ho to enable it.

  1. Set logging level of logger de.rtb.pcon.config.general.WebIntegration to trace.

  2. Set property pcon.development.debug-web-client to one of following value:

    1. off - Logging disabled. Use in production.

    2. simple- Log only events.

    3. text - Both events and content will be logged, with content in plain text format.

    4. hex-dump - Both events and content will be logged, with content in hex format.

Logging is possible only for partners which uses new WebClient.

Message forwarding improvements

E-mail addresses entered to Recipients field in message forwarding configuration can contain optional prefix determining for what field the e-mail address will be used. Allowed prefixes are to:, cc: and bcc:. If prefix is used then the email address will be used inside corresponding address header. E-mail addresses without prefix are assumed to be to:.

From this version on, if there are multiple addresses configured in the message forwarding rule, only one e-mail is sent to all recipients. Previously each recipients got a single e-mail.

All e-mails with prefix
to:administrator@example.com,cc:parking-helpdesk@rtb-bl.de,bcc:user@example.com

As to: prefix can be omitted, it is also possible to write address list like this.

Default to:
administrator@example.com,cc:parking-helpdesk@rtb-bl.de,bcc:user@example.com

Using of phone numbers for SMS stays unaffected, therefore it is still possible to mix them with e-mails

Mixed recipients
+491231456789,administrator@example.com,cc:parking-helpdesk@rtb-bl.de,bcc:user@example.com

Other improvements

  • Changed ordering of 'parking processes'. Now they are ordered by arrival time and then by ticket number.

  • Configurable integration with WES.

  • Timestamp database indexes reverted to B-TREE. Database update may take longer then usual. See PCON-2674.

22.1.1

  • Stabile order of currencies in cash box level.

  • Reset form after remote action start.

  • Documentation updates.

22.1.2

  • Fixed problem on the map if there are no PDMs with an issue.

  • Fixed PDM availability monitoring.

22.1.3

  • Updated Brain Behind endpoint.

22.1.4

  • Fixed mobile network type indicator on PDM page.

22.1.5

  • Updated format of report "Ticket sale".

22.2.0

  • Added summary to report "Amount by payment type"