EPICS v4 and EPICS7

In a previous post I introduced the EPICS control system and in practice what I was describing was EPICS v3. This has served Diamond well in the 15 years since the Diamond control system started development, but in recent years there has been a lot of discussion in the EPICS community about EPICS v4 and more recently EPICS7. What is the difference, and why is it a big deal?

In practice, EPICS v4 was a fundamental change to EPICS, and although it evolved over time what came out was really something different to what is available in EPICS v3. It took many years for the community to understand how v4 would properly integrate into EPICS. Eventually it was decided that EPICS v3 and EPICS v4 would essentially be combined into a larger toolkit dubbed EPICS7. All future versions of EPICS will include both parts.

What was wrong with EPICS v3?

There wasn't really much wrong with EPICS v3, and that is why it has built such a robust community. That said, over time, a number of limitations were found with the control systems described in my earlier post:

Synchronisation: what if you want to control two PVs so that they reliably change at the same rate and at the same time? Over time, a number of ways were designed to work around this, but fundamentally EPICS did not offer a way to do this.
More sophisticated data types: EPICS provided very simple data types. This is easy to reason about, but limits how well you can manage more complicated data types. IOCs provided a very large number of PVs with closely-related information, but these could be difficult to work with. Things like 2D arrays were encoded in 1D arrays, with the understanding that the client would have to reshape the data back to how it ought to be.
Efficient data transfer: channel access is efficient as far as it goes, but it is not really designed to transmit large quantities of data over the network, even if applications such as areaDetector did do just this using large 1D arrays.

What did EPICS v4 add?

The most obvious change in EPICS v4 was the addition of structures. A PV could now contain a tree of related information: for example an array of doubles and some additional metadata about that array. These structures update atomically - that is, you can rely on the updates to the different part of the structure arriving at the client at the same time.

This more sophisticated data transfer necessitated the replacement of channel access with a more sophisticated network protocol. This was called pvaccess, and a lot of effort was put into ensuring that this protocol not only reliably supported these new data types, but that it could efficiently use the network to transmit data as quickly as possible.

The data types also required programming language implementations; these too were heavily optimised to ensure that (for example) unnecessary copies of large amounts of data in memory could be avoided. Client toolkits were added in C++, Java and Python to enable custom pvaccess clients to be developed.

Another topic that came up was the idea of generic (in a programming sense) services: how could you support an arbitrary request, where you provide some input, something does some calculation and then returns you a result? A common name for this is a Remote Procedure Call or RPC; this was also added to v4.

Finally, the freedom to define arbitrary structures does present one particular problem: a client needs to understand the structure provided by the server in order to do something sensible with it. This problem is mitigated by the development of normative types, a set of predefined data structures that are likely to meet the needs of typical control systems. A client receiving one of these normative types has a better chance of understanding what to do with it than if it had received an arbitrary structure.

NTScalar :=

structure scalar_t value string descriptor :opt alarm_t alarm :opt time_t timeStamp :opt display_t display :opt control_t control :opt

The simplest normative type, the NTScalar, just to give an impression of what these structures look like.

What happened next?

Once these new features were available, a number of questions began to arise. How would we connect these new datatypes to hardware, as we had in EPICS v3? What about all the drivers and IOCs we were already using in EPICS v3? How might a facility migrate to using v4 and would it be worth it?

I don't know how early on this decision was made, but eventually it was realised that v3 itself would not be replaced. Instead, the new features of v4 could be added to those of v3, meaning that all existing databases and IOCs could be used more-or-less as they are. In the case that you want your old IOCs to be used with your new pvaccess client tools, a shim was developed to provide that same information over pvaccess as well as channel access.

The Git repositories of the original EPICS base and the new v4 code were merged into the new EPICS7 repo, and the build was combined. The first version of EPICS 7, version 7.0.0, was finally released in the summer of 2018.

So what now?

A number of facilities are now using EPICS7 in production, but what should facilities like Diamond do? In practice our control system is designed around EPICS v3, and the benefits of EPICS7 are more theoretical than pressing.

In practice, the decisions described above make it easy for us. When we upgrade our 'v3' version of EPICS (itself a major task), at some point we will get 'v4' included for free. We plan to install a version of EPICS7 later this year, and then our developers will be free to explore the new capabilities that will be available.

I don't think that this is something to rush. Although EPICS7 contains a larger number of tools, there are other tools that can serve the same purpose and some may do so better. There's no need to use new EPICS features because there are new EPICS features. Over time, Diamond will build a control system that uses the full capabilities of EPICS7, and we should be able to do this in time that we are ready to exploit its full potential for Diamond-II.