Last revision: March 2021

ICM Workgroup setup best practices

Guidance for optimal performance in a workgroup environment

1

Contents

Introduction ........................................................................................................................................................................................................................ 2

Workgroup environments ..................................................................................................................................................................................... 3

Collaboration Type ................................................................................................................................................................................................. 4

Location ........................................................................................................................................................................................................................... 6

Specialty .......................................................................................................................................................................................................................... 7

Modelling machines ........................................................................................................................................................................................ 8

Server Machines ................................................................................................................................................................................................. 9

Simulation machines ..................................................................................................................................................................................... 11

Data management ..................................................................................................................................................................................................... 12

Running simulations concurrently ................................................................................................................................................................ 13

Hyper-threading (HT) ............................................................................................................................................................................................... 15

2

Introduction

This document provides information on the roles and relationships of the workgroup components and best practices during the implementation of a workgroup solution. The suggested topologies are not prescriptive, since the main goal is to guide the reader to a solution that fits their needs. Therefore, some terminology is deliberately qualitative in nature.

A Workgroup is the best method of deploying InfoWorks ICM robustly, from single users to large collaboration user groups. It is designed for flexibility, so configurations can be tailored to the needs and constrains of the end-users and their organizations, informed by understanding the roles of each product. These depend on factors such as (but not only):

• Models: size, type (1D/2D), number

• Simulations: number, frequency, duration, size

• Users: number, geographical location

• Data: importance, sensitivity

• IT: network infrastructure, budget constraints, security constrains

Before proceeding, the reader is expected to be familiar with the Innovyze Workgroup Products IT Architecture documentation. Terminology detailed under Key Components and Key Concepts will be used here. The full document is available from the Innovyze Support Portal.

3

Workgroup environments

Workgroup environments are comprised of machines that contain components which can perform distinct roles. For example:

• A machine whose role is to allow access to the modelling graphics user interface needs the Workgroup Client component installed.

• A machine whose role is to manage a workgroup database needs the Workgroup Data Server component installed.

• A machine whose role is to run simulations needs an Agent component installed.

• A machine whose role is to act as a coordinator needs an Agent component installed.

Workgroup roles can be performed by different machines according to the needs of the end users: from centralized configurations where one machine performs most Workgroup roles, to decentralized configurations comprised of specialist machines.

The example machines in this document will be split based on Collaboration Type, Location and Specialty. This is an artificial categorization, aimed at simplifying the roles of the Workgroup components.

It is worth bearing in mind the rules around versions for software upgrades. For more information check this online article.

https://innovyze.force.com/support/s/article/InfoWorks-ICM-Version-Numbers-The-rules

4

Collaboration Type

A Workgroup environment allows two types of collaboration:

• Modelling collaboration - by using a Workgroup Database, which allows users to contribute simultaneously to a single database, with versioning and conflict resolution entirely managed by the software. This is managed by the Workgroup Data Server component.

• Computational collaboration - by using a Coordinator, which manages a pool of machines that are available to run jobs on demand as part of one computational resource pool. This is managed by an Agent component acting as a Coordinator Agent.

Machines in a workgroup can make use of these systems independently. For instance:

• A Workgroup Data Server can be used for modelling collaboration without using a Coordinator Agent for distributed simulations.

• A Coordinator Agent can be selected to take advantage of a computational pool whilst using a standalone database.

This diagram below highlights the different collaboration types:

5

Modelling collaboration Computational collaboration

For example, users have machines with InfoWorks ICM that connect to a central model repository to obtain modelling data. They can then contribute to the modelling repository by submitting their local versions.

To use the Innovyze terminology, Workgroup Client users connect to the Workgroup Data Server to obtain the latest Local Working Files. Once user finish changes to the Local Working Files, these are committed to the database, updating files under the Data Store.

For example, instead of running in their own machine, a user can request simulations to be run by a pool of remote machines. These machines need to be connected to a central node that manages the computational pool.

To use the Innovyze terminology, a user can instruct their Agent (as Local) to send jobs to other Agents (as Remote) to be processed. All Agents need to be connected to a central Agent (Coordinator) which coordinates jobs between participating machines.

6

Location

The workgroup solution is designed to allow a level of portability. For instance, a user can do modelling work on a laptop in multiple locations. They only need a connection to a Workgroup Data Server for version control and/or a Coordinator for to run simulations remotely. Modelling Machines, but other workgroup machines benefit from being in a static environment:

• User Space – comprising machines used by modelers for modelling work, allowing flexibility to access the workgroup from multiple locations (such as at different offices or at home).

• Server Space – comprising backend machines that share a static environment with reliable interconnectivity.

Note: Modelling Machines need a reliable and broad connection to the Remote Root folders to access remote files. In office configurations, gigabit connections

are recommended.

Working with remote results files over a slow or unreliable connection (such as a typical home internet connection) might make remote results analysis unfeasible. IT systems administrators can consider installing Workgroup Client machines in the same environment as the Remote Results Root. Users can then connect to the Workgroup Client from low bandwidth locations, using protocols such as Citrix or Remote Desktop. This is only possible providing an appropriate Floating License is used.

7

Specialty

Workgroups can benefit from specialized machines that perform only specific workgroup roles. This is a good strategy to ensure best performance of those roles whilst optimizing hardware specifications.

In this example, machines will be grouped under one of the following three categories: Modelling, Server, and Simulation.

The following sub-sections cover three types of machine specializations.

8

Modelling machines

These are used to run the ICM graphics interface client alongside other supporting applications.

Typical tasks include updating modelling networks, committing changes to modelling databases, running occasional simulations, and analyzing / processing results.

They should have above average and balanced specifications compared to a typical end-user machine:

• Fast CPU. Some ICM processes (such as simulations) rely on raw computational performance, which is proportional to CPU clock speed. However, this metric alone should not be used to compare between processors with different architectures.

• CPU with multi-core capability. Mostly to allow seamless multitasking with other installed applications. The simulation engine can also leverage multiple CPU threads to improve simulations speeds.

• High amount of RAM. The ICM graphics interface and the ICM simulation engine do not use a high amount of memory, except in exceptionally large models or when loading large GIS background files. However, these machines often run other applications which use memory themselves, and low available memory will impact negatively in the overall performance.

• SSD storage. The disk will need to host the software, local copies of the modelling files and local results files. ICM will perform best in disks with fast access speeds, particularly for tasks such as results files analysis. These files can be cleaned, so a large disk might not be required. It should however be proportional to the modelling work which is expected to take place.

• CUDA enabled NIVIDIA graphics card (Optional). Only relevant if running models with 2D components. Depending on the 1D/2D ratio, it can improve simulation speeds dramatically.

Software: Modelling machines will only require the Workgroup Client software.

9

Server Machines

These machines carry out the server roles of managing modelling databases, storing modelling simulation results, and coordinating the Workgroup environment. Tasks performed by machines running the WDS or Coordinator Agent background services are lightweight, so server type CPUs with average clock speed/cores are appropriate. This should be reviewed if large models or vast numbers of simultaneous users are expected to access these services.

Workgroup Data Server (WDS)

The role of the WDS is to manage the Workgroup Databases under the Data Store. This server requires enough storage space and memory for hosting and accessing the database data:

• Storage: For best performance and reliability, the directory used to store standard databases (Data Store) should be local to the machine running the WDS service. There should enough space to host all data – this includes networks, ground models, time-varying-data, and TSDBs. Faster drives (such as SSDs) might also improve performance in servers that are heavily accessed. By default, the WDS installer chooses the location ‘C:\ProgramData\Innovyze\SNumbatData’ for the Data Store.

• Memory: Minimum 4GB, recommended 16GB. The machine should have more RAM than the size of the largest network database (.wdb) file on the server.

More information on the minimum specifications for this service can be found on the Workgroup Data Server Administration document available for download on the Innovyze Support downloads page for InfoWorks ICM.

Note: The same machine can be used for both the WDS and Coordinator Agent roles, since these services are lightweight. However, it is good practice to use

different disks for hosting the Data Store and the Remote Results Root. See Data Management section below for more information.

Software: The WDS requires the Workgroup Data Server software installed, which does not need a license.

Coordinator Agent

The role of the Coordinator Agent is to manage the computational resources in a Workgroup environment. It directs traffic related to remote ICM jobs (such as simulations or meshing) between connected Agents.

The Coordinator Agent manages the flow or results data from a Remote Agent to a Remote Results Root. To ensure the data flow of large results is not throttled, there should be a good network connection between the Coordinator Agent machine, the Remote Agent (running the simulations), and the Remote Results Root (where the results are stored). For example, by keeping the Remote Results Root as a drive local to the Coordinator Agent, and the Remote Agents in the same Server Space.

10

Software: The Coordinator machine require the Agent service to be installed. Both the Workgroup Client and the Remote Agent installers contain this service. If this machine needs a user interface for ICM, the Workgroup Client should be installed. If it does not, the Remote Agent should be installed. This role does not require a license.

Remote Results Root

The Results Remote Root should be a large, central, and scalable storage space. Workgroup Client machines access this location when analyzing remote results. Simulation results files are significantly larger than modelling databases.

Note: The performance of Workgroup Client tasks that require access to files in the Remote Root – such opening a simulation, graphing results, producing statistical templates, loading shared GIS files – will depend on how fast the Workgroup Client can access them. In practice, this means that it is faster to process results stored in a local drive, and it could be impossible to work on

exceptionally large results if the access speeds are slow.

Examples of good practice when storing results:

• A fast and local internal drive will be accessed much faster than from a remote server over an internet connection.

• Internal drives connected via a bus interface such as SATA 3.0 have transfer rates around 20x faster than an external USB 2.0.

• Solid State Drives (SSDs) have read speeds at least 5x faster than Hard Drive Disks (HDD).

• RAID 0 or 10 configurations can theoretically double read speeds

Results data growth is difficult to predict since it is relative to the complexity of the models and duration/number of simulations. Old results should be maintained (archived or deleted) on a schedule to prevent a results drive running out of disk space, potentially causing ongoing simulations to fail.

Software: No software is required, ICM products will use Windows file sharing protocols to access the Remote Results folder. User permissions should be granted for access by all relevant machines.

11

Simulation machines

These are specialist high specification machines designed to do the heavy lifting of running computationally intensive simulations:

• CPUs with best possibly benchmarked performance. The better the CPU performance, the faster simulation run times will be.

• CPUs with a high number of cores. Simulation related computation can be spread between multiple cores.

• NVIDIA GPU CUDA enabled cards (optional). This is relevant only for 2D intensive models. There is specific documentation about this topic which can be requested from Innovyze support.

• Ample disk storage. Enough disk space to cache all results temporarily in the Remote Agent Working Folder before they are sent to the Remote Results Root. Particularly relevant when running multiple concurrent simulations.

• Fast disk storage. Relevant when running multiple concurrent simulations to prevent throttling from several results files being simultaneously written to the disk.

• High amount of RAM. Relevant for meshing jobs and when running multiple concurrent simulations since each simulation will allocate memory individually.

Note: ICM simulations can use all available CPU resources (if unrestrained) and can increase disk operations (when saving results). It is good practice to install

remote Agents in machines that are not running critical processes. For instance, the Workgroup Data Server or the Coordinator Agent services can be affected if

the same machine is also running simulations, and this has the potential of impacting on the whole workgroup environment. Machines used for simulations

will also put higher stress to their hardware components.

Software: Simulation machines require the Agent service, which is installed as part of the Workgroup Client or Remote Agent installers. This Agent uses licenses when running jobs, which need to be configured when running remote jobs on behalf of other Agents.

Simulation machines can run jobs for multiple versions of ICM, providing they have the same Remote Agent version as the Workgroup Client requesting the job. The version needs to be exact down to the minor patch number. The latest version of the Agent Manager should be running the service since it is backwards compatible with earlier engines.

Example: If using Workgroup Client version 9.5.1 to run simulations remotely, the Simulation machine must have version 9.5.1 of the Remote Agent installed. Version 9.5.5 would not work, and one machine can only have one minor version - 9.5.1 and 9.5.5 cannot be simultaneously installed. However, the simulation machine can have 9.5.1 and 11.0.2 installed, since they are different major versions. The Remote Agent can have version 11.0.2 of the Agent Manager running, since it can manage the older 9.5.1 Agent.

12

Data management

There are strategies to mitigate the risks of data loss through hardware failure, corruption, or unintentional changes. These include

• Data redundancy – On-the-fly duplication of data in multiple disks in case one fails, such as RAID.

• Backups – Archiving data at regular intervals for future recovery. This can be as simple as scheduling a recurrent time stamped copy task to an archive drive, but there is also specialist software available to manage backups.

Database files stored in the Data Store and remote simulation results files stored in a Remote Results Root are different, and there are benefits in separating them into different drives:

• Compartmentalizes data storages. If a storage system has issues, (for instance, simulation results filling up a drive) it will not affect the performance of the other storage system.

• Can reduce the volume of data that needs redundancy or backup. Modelling data is more critical and onerous to recreate; results data can be regenerated by rerunning simulations.

13

Running simulations concurrently

The InfoWorks ICM engine can both split a simulation between multiple computational threads and spread multiple simulations across multiple threads.

Running a simulation using multiple threads can improve the total time it takes to complete it. However, speed gains diminish as more threads per simulation are used. After several threads are used, improvements in performance become negligible and could even decline. In other words, there are limits to performance improvement when increasing thread count per simulation. This is caused by the various threads competing to access the same memory block simultaneously.

However, it is possible to optimize multi-core processors by running various simulations concurrently. Individual simulation speeds might be slower, but more simulations can be run at a higher performance per simulation. However, disk size and memory need to be scaled up to accommodate larger numbers of simulations running concurrently.

In an example where a machine with 56 available cores is running 56 concurrent simulations:

• If each sim uses 1GB of modelling data, it will require at least 56GB of free RAM.

• If each sim produces 10GB of results per simulation, it will require a disk with at least 560GB of free space.

• 56 simulation results would be written simultaneously, which increases disk operations. Drive access speed might bottleneck the output of processed results.

Alternatively, running 12 simulations concurrently at a ratio of 1 sim per 4 cores could be a more balanced approach.

The number of used computational threads can also be explicitly restricted:

• For all runs: by opening the Agent Manager via Agent options - Performance – Max Threads Total. This requires restarting the Agent.

• For a specific run: in ICM when starting a simulation run after pressing Run simulations – multi-threading limit – Limit threads-per-job to.

14

If the number of threads is not restricted, ICM will use all threads and split them between simulations as they become available. For example:

• A machine has 12 available threads.

• A set of 6 concurrent simulations is set to run in parallel.

• Each simulation starts by using 2 threads each.

• When 4 simulations finish, 8 threads become available.

• The 2 remaining simulations are split among the 8 newly available threads.

• The 2 remaining simulations now use 6 threads each.

Note: The simulation log file will only show the last number of threads used by the simulation when it ends. In the example above, this means the first 4

simulations would show 2 threads in the log file, and the last 2 simulations would show 6 threads in the log file.

15

Hyper-threading (HT)

This virtualized technology allows one core to be shared between two threads. Testing suggests that HT does not improve simulation performance for ICM modelling. Whilst it doubles the number of available threads, the ICM engine does not benefit from this as it runs at half the speed per tread. In some cases, disabling HT might improve performance:

• Processes that can only use one computational thread – such as meshing or pre- processing – will have access to a full core instead sharing a core with another thread.

• Running more concurrent simulations to ‘take advantage’ of more computational threads will not improve run times, and it will increase the demand on other hardware components, since the machine will be processing more simulations simultaneously. This could be in the form of insufficient memory, or not enough storage space.

• In machines with a vast number of available threads (>64), a small number of cases were reported where ICM was only able to use half the total number of threads, effectively halving the computational output of the machine. Disabling HT halves, the number of available threads reported by Windows, which in turn allowed ICM to fully utilize the available computational resources.