Custom data interfaces with external applications – Applying middleware that interacts with the Relatics API

Introduction

An often seen situation is that a project team uses Relatics to manage requirements while at the same time managing their documents in a different application, typically a document management system (DMS). However, if requirements are managed in Relatics and documents are managed in a DMS, how can we keep track of which requirements come from which source documents? After all, we want to instantly know which requirements are involved and should be re-analyzed after a new version of a source document is provided.

Fortunately, Relatics offers a solution: application integration. Data interfaces allow Relatics to be synchronized with external applications. In our example, this results in each document in the DMS having a counterpart in Relatics in the form of a Relatics element. Now, Relatics users are able to record relationships between requirements and documents.

The objective of this tutorial is to explain data interfaces between Relatics and external applications and how middleware plays a critical role.

Relatics API & middleware

Relatics offers dedicated interfaces to connect with specific external applications (refer to Create an interface with an external application.) However, your project may use an external application for which there is not yet a dedicated interface available. As an alternative, you can set up your own custom data interface by applying middleware that invokes the API of Relatics.

The image below gives an overview of the interaction between Relatics, the external application and the middleware.

1. As the first step of a communication ‘loop’, the middleware requests data from the external application.
2. The external application responds by returning the requested data. The middleware receives the data in the format specified by the external application’s API.
3. Middleware ‘translates’ the received data into the format of the Relatics API.
4. Now that the data is in the right format, it can be forwarded to Relatics. Here, the communication iteration ends.

How Relatics elements are created, modified or deleted is the responsibility of Relatics; the middleware has no role in this. Thus, middleware is nothing more than a piece of software that acts as an intermediary between Relatics and an external application. Its primary two responsibilities are 1. Orchestrating the communication between Relatics and the external application and 2. Transforming data between different formats. From a technical perspective, the middleware operates as a client of the external application’s API as well as a client of the Relatics API.

SharePoint & Power Automate

To keep things as tangible, intuitive and understandable as possible, the remainder of this tutorial uses SharePoint as the external application with which to interface. The reason for this is that SharePoint is commonly used and known. At that, SharePoint is available to most of us.

Using SharePoint as our running case, it is straightforward to use Microsoft Power Automate as middleware. After all, both products have the same vendor and a single Microsoft account can be used for authentication in both products. Refer to Microsoft Power Automate documentation for more information about Power Automate.

Tutorial structure & reading tips

In the next section, we set up the middleware to fully interact with Relatics. Authentication is an important aspect herein. After all, we only want our own middleware accessing the Relatics API; any attempt by other actors should be blocked. In the subsequent section, we complete the data interface by extending the middleware with the interaction with SharePoint, and take care of the difference between Relatics’ data format and SharePoint’s. In the sections thereafter, the data interface is extended. At that point, it already controls elements and properties, but then it will also control relations in Relatics.

It cannot be emphasized enough: this tutorial was never meant as a recipe for connecting Relatics with SharePoint. There is a very useful native integration for that. So please resist the urge to get stuck into the nitty gritty details on first read, and try to focus on the ‘larger picture’ instead. Read through the steps, try to understand the idea behind a step without getting overwhelmed by details. If, however, after one or more reads, you still fancy getting hands-on experience, you are more than welcome to follow along with the steps.

Before you click along with this tutorial, you might want to take care of the following in advance:

• Do you have the Administrator role in the Relatics environment? Refer to Create users in the environment.
• In the Relatics workspace, are you a Power User? Refer to Adding users to a workspace.

Provide documents to Relatics – A first step

Prepare in Relatics

1. We need a type element to enable Relatics to synchronize with an external application. Therefore, we create type element [Document] with Ownership in set to External application. The setting Ownership in controls whether Relatics users create elements from within Relatics, or whether the creation of Relatics elements is delegated to an external application. In our case, we want the creation of [Document] elements to be delegated to SharePoint. Therefore, we set Ownership in to External application.

2. Given a [Document] element in Relatics, we want to recognize which document in SharePoint it corresponds to. With that in mind, we also set Ownership in of type property {Filename} to External application. Similar to the previous step, the idea is that not the Relatics users fill the {Filename} properties, but that Sharepoint does so instead.

3. Obviously, external elements have relations with internal elements. For the remainder of this tutorial, take care of at least one relation, for example with type element [Requirement].

4. So far, we have prepared the type model in Relatics. But before an external application can create elements and fill properties in Relatics, access has to be granted first. Similar to how user access is maintained at the environment level, external application access is also managed at the environment level. Via Environment administration | Application authentication, create a new Relatics API key for requests by external application(s).

5. Fill in a name (for example, “Provide documents by SharePoint”) and save a copy of both the “Client ID” and the “Token endpoint URL” (for example, by pasting them into Notepad). The latter is handy in a later step of this tutorial.

6. After the Create-button, a dialog pops up giving you the one-time (!) opportunity to obtain the client secret. It is important that you temporarily save a copy.
   
   NB: after the Close-button, there is no way to retrieve the client secret from Relatics again.

7. After taking care of the authentication on the level of the Relatics environment, we are ready to set up the data interface on workspace level.

8. Fill in the settings as below.
   
   Note that you selected the Type element (with ‘Ownership in’ an external application) that was created in step 1 and the Relatics API key for requests by external application that was created in steps 3-5.

9. Don’t forget to save a copy of the Interface endpoint URL; this will be handy later in this tutorial.

Authenticate on Relatics

Now that authentication on Relatics’ side is taken care of, it is time to set up Power Automate to authenticate on Relatics.

10. In Power Automate, create a new flow: Provide documents to Relatics.

11. For the time being, choose for the manual trigger.

12. Add a new action.

13. Be careful to choose HTTP (which requires a Premium license of Power Automate).

14. Give in a name like “HTTP – Authenticate on Relatics”, paste the Token endpoint URL that you copied in step 4 into the URI parameter and choose POST as the method.

15. Include a Content-Type header with the text: application/x-www-form-urlencoded and include a second header Authorization filled with the text: Basic.

16. For the complete Authorization header, we have to encode the client ID and the client secret. Therefore, open a separate browser tab and go to Base64encode. Copy Client ID from step 4 and Client secret from step 5, separated by a colon and click ENCODE.

17. Copy the encoded string from base64encode.org and paste it into the Authorization header in Power Automate.

18. Specify the Body as “grant_type=client_credentials”.

19. Save the flow and test it.

20. When we check the output of the response of Relatics, it should look like:

• The fact that Relatics provides an access token implies that the authentication on Relatics was successful.

Request to Relatics API

Provided with an access token, Power Automate can now access the Relatics’ data interface for synchronizing that we set up in steps 6 and 7.

21. To achieve this, we create a subsequent HTTP Post request “HTTP – Provide documents to Relatics”.

22. Paste the Interface endpoint URL that we copied in step 9 into the URI parameter and choose POST as the method.

23. Specify Headers as below:

Accept	application/json
Content-type	application/json
Connection	keep-alive
Authorization	Bearer @{body(‘HTTP_-_Authenticate_on_Relatics’).access_token}
Host	[yourenvironmentname].relatics.com

• Note how the code in the Authorization header retrieves the access token from the output of the previous HTTP Post and includes it in the Authorization header behind the text: Bearer.

24. Match the Host header with the first part of the URI pasted in previous step 22.

25. With the purpose of testing the communication between Power Automate and Relatics, we temporarily ‘hard code’ two fake documents. In the Body field, copy the following JSON.

{  “Documents”: [   {    “@@externalid”: “12345”,    “Filename”: “test1.docx”   },   {    “@@externalid”: “67890”,    “Filename”: “test2.pdf”   }  ] }

• @externalid is a system field that should be provided in any data interface. Else, Relatics can not decide between creating a new element versus modifying an existing element. (The reason that “@” is included twice is because “@” has special meaning in Power Automate and we have to escape it. Refer to the Workflow Definition Language, at the bottom of section Expressions).
• Filename is a field that corresponds to the {Filename} property we defined in step 2.

26. Test the flow in Power Automate and verify the [Document] elements in Relatics. When documents arrive correctly in Relatics, this proves that Power Automate is able to manipulate [Document] elements in Relatics.

Pass on documents – A minimal data interface

In the previous section, we managed to let Power Automate provide documents to Relatics. However, these documents were ‘hardcoded’ and SharePoint was not involved yet. In this section we let Power Automate actually act as middleware, in that it requests SharePoint for its set of documents and passes these on to Relatics.

Prepare in SharePoint

27. Identify the Site Address and the Library Name of the documents in SharePoint.

Request to SharePoint API

28. We create a new action Get files (properties only) in front of the flow.
    
    Because we use the same Microsoft account for both Power Automate and SharePoint, Power Automate can access SharePoint without further bothering about authentication.

29. Fortunately, Power Automate offers templates for accessing SharePoint.

30. This makes it super easy to set up; specifying Site address and Library name is only a matter of selecting from the according drop downs.

31. Because we are interested only in the documents and not their folders, we specify the following in the Filterquery field. 

FSObjType eq 0

32. Test the flow and inspect the output of Get files (properties only). The body shows how SharePoint responded to the request, which should look something like the example below. As you can notice, SharePoint responds by listing all the documents it contains.

Map SharePoint data onto Relatics fields

33. The format in which SharePoint specifies its documents is different from the format in which Relatics wants [Document] elements to be offered. In other words, we have to convert the information from SharePoint into the Relatics format. We achieve this with another action.

34. From Data Operations, choose the Select template to create a new action.

35. We specify from which part in SharePoint’s response we should extract information.

36. We map SharePoint fields onto Relatics fields. That is, SharePoint’s field ID is the source for Relatics’ field @externalid. Similarly, SharePoint’s field {FilenameWithExtension} is the source for Relatics’ field Filename.

37. Now that the data received from SharePoint is transformed into the Relatics format, we can send it to Relatics. Therefore, we replace the (fixed) testing data we entered in the body in step 25 with the output of the Select action.

38. Test the flow in Power Automate and verify the [Document] elements in Relatics.
    
    As intended in step 6, we now have an interface that synchronizes Relatics with SharePoint as the external application. For each document in SharePoint, Relatics has an equivalent [Document] element. But at the same speed, documents that are not in SharePoint, will not be in Relatics either. Thus, the test documents of step 21 were deleted from Relatics during synchronizing.

Complete the data interface

39. In the previous steps, we enabled users in Relatics to create relations between elements that are managed in Relatics for example requirements and elements that represent the documents in SharePoint.

40. The image above is a screenshot of how a user in Relatics experiences a [Document] element. However, users will also have a desire to view the equivalent document in SharePoint. To facilitate this, we define an extra type property {SharePoint link} with DataType=URL and Ownership in set to External application (similar to step 2).

41. Then we extend the mapping of step 36 to:

42. As a result, users in Relatics obtain an URL that brings them directly to (the preview of) the document in SharePoint.

43. Finally, we want to have the flow running automatically, instead of the need to start it manually over and over. This is achieved with a recurring action. For example, if we want the middleware to operate once in every 3 minutes, we can specify this via the Interval/Frequency settings.

Final thoughts

As explained in the introduction, this tutorial is not titled “Recipe for connecting Relatics with SharePoint”. In fact, if your project has a specific desire to interface with SharePoint, you are strongly advised to rely on the dedicated SharePoint-interface that Relatics offers off-the-shelf (refer to Application integration using SharePoint REST). After all, a dedicated data interface is set up in a couple of clicks. Moreover, dedicated data interfaces offer dedicated functionality. In case of the data interface dedicated to SharePoint, the Relatics user can preview a SharePoint document without leaving Relatics. Furthermore, a user can filter on [Document] elements based on their according folder in SharePoint. The only reason why this tutorial utilizes SharePoint as the case at hand, is that it makes the material easy to digest.

As a rule of thumb: if Relatics offers a dedicated interface with an external application, try that first, before deciding to use the alternative method as laid out in this tutorial. On the other hand, if your project desires Relatics to interface with an external application for which there is no dedicated solution available yet, this tutorial is a good starting point. The ideas covered can be widely applied. Think about custom interfaces with for example GIS-, planning- or HRM-applications.

Depending on the constraints and desires in your project you might want to choose another middleware tool than Power Automate. For authentication on Relatics, you should follow the OAuth 2.0 protocol. Your middleware tool may support this protocol, which would relieve you from worrying about technical stuff like expiration of access tokens, refresh tokens etcetera. And probably, the client ID and the client secret can be specified in a more discrete way compared to the ‘transparent’ way of hardcoding client ID and client secret in steps 16 and 17. After all you can (and should) consider these ‘credentials’ just as sensible as usernames and passwords, and the right combination enables anyone to manipulate data in Relatics. With full support of OAuth2.0 you can imagine that the client secret is input, while the screen (or code) shows only asterisks ‘*’.  When choosing from all the middleware possibilities, consider to what extent the OAuth2.0 protocol is supported.