Parser Component

:icons:font :hardbreaks: :imagesdir: ../../../assets/images :toc-title: Contents :toc:

Description

Parses any text structures. The regular expression method, the JSON search method, and the built-in hypertext analyzer can be used. According to the search string, it generates a response, either cuts out the requested section or a set of them, or determines the number of elements in the specified section of the structure. Complex structures can be parsed by a sequence of "Parser" elements, each of which extracts a certain structure from the document and returns it for transmission to the input of the next element.

The parsed document structure is cached in a specific instance of the script engine to speed up work with one large document of sequential Parser elements.

Table 1. System characteristics

Index

125

Short name

parse

Script types

All

Start module

r_script_component_parse

Mode

Synchronous

Icon

125

Branching type

Branching, closing

Properties

Table 2. Properties
Specification Description

Name: Document

Code: data

Visibility condition: no

Default: — 

Argument with the contents of the document to be parsed.

Name: Algorithm

Code: algoritm

Visibility condition: no Default: Regular expressions

Parsing algorithm.

Possible options:

  • json parser (json, 0) – Applies a search algorithm in a JSON structure (object or array). Examples of search queries below.

  • XML parser (xml, 1) – Applies a search algorithm in an XML document. Examples of search queries below.

  • Regular expressions (regular, 2) – Applies the standard regular expression algorithm. Searches for elements using a search pattern. In general, it detects several elements. To display content, you must specify the element number and group number (if the template uses group capture).

  • HTML parser (html, 3) – Applies a search algorithm in an HTML document. RQuery syntax. Examples of search queries below. Documents differ from XML in that they lack a strict document structure, namely, they allow for the absence of closing tags.

Name: Search query

Code: query

Visibility condition: no

Default: — 

String with the search query for the selected algorithm.

Name: Function Code: functionJSON Visibility condition: yes

Default: Number of elements

Function for the JSON Parser algorithm.

Possible options:

  • Content (content, 0) – Returns the detected element (the content that has not been decoded). In JSON text representation, any string value must be encoded using escape sequences. The return value contains escape sequences.

  • Recovered content (restoreContent, 1) – Returns the detected element (the content that has been decoded if it is a string). The returned string value is already stripped of escape sequences, so running unescape again is unnecessary and can be harmful since the original string content itself may also contain recognizable escape sequences.

  • Number of elements (count, 2) – Returns the number of elements found.

  • List of keys (keys, 3) – Returns a list of keys in the found object.

Name: Function

Code: functionXML

Visibility condition: yes

Default: Number of elements

Function for the XML Parser algorithm. Possible options:

  • Document (textContent, 0) – Returns the entire selected element along with its contents.

  • Content (content, 1) – Returns the contents of the selected element, excluding the element itself.

  • Number of elements (count, 2) – Returns the number of elements detected.

  • Attribute Value (attributeValue, 3) – Returns the value of the specified attribute of the selected element.

  • Attribute Name (attributeKey, 4) – Returns the name of the attribute by its index in the selected element.

  • Number of attributes (attributeCount, 5) – Returns the number of attributes in the selected element.

Name: Function

Code: functionREGULAR

Visibility condition: yes

Default: Number of groups

Function for the Regular Expressions algorithm. Possible options:

  • Content (content, 0) – Returns the contents of the specified group from the specified element. Counting elements with 1 (default value), counting groups with 0 (default value 1), with the zero group being the complete contents of the element. The use of groupings (parentheses) in the search template entails the need to specify the correct value of the group of interest; specifying an inappropriate index results in the component terminating with an error.

  • Number of Elements (countElements, 1) – Returns the number of elements found that match the search pattern.

  • Number of Groups (countGroups, 2) – Returns the number of captured groups in the detected element at the specified index. The element number is required, by default 1. If no grouping is used in the search pattern, the value returned is 0.

Name: Function Code: functionHTML Visibility condition: yes

Default: Number of elements

Function for the HTML Parser algorithm.

Possible options:

  • Document (textContent, 0) – Returns the entire selected element along with its contents.

  • Content (content, 1) – Returns the contents of the selected element, excluding the element itself.

  • Number of elements (count, 2) – Returns the number of elements detected.

  • Attribute Value (attributeValue, 3) – Returns the value of the specified attribute of the selected element.

  • Attribute Name (attributeKey, 4) – Returns the name of the attribute by its index in the selected element.

  • Number of attributes (attributeCount, 5) – Returns the number of attributes in the selected element.

Name: Item number Code: elementNumber Visibility condition: yes

Default: — 

Parameter for the Regular Expressions algorithm. Specifies the index of an element in the list of elements found by the pattern in the source document.

Name: Group number

Code: groupNumber

Visibility condition: yes

Default: — 

Parameter for the Regular Expressions algorithm. Indicates the group number in the list of captured groups of the selected item.

It only matters when you specify capture groups in the search query (set off in parentheses in accordance with regular expression syntax).

For example, in the document, mom was washing the frame, and Lena was sitting, the template (.)a([^ ])* will find 7 elements, in each of which 3 groups are available: 0, 1, 2. For the element 4, the group 0 is frame, 1 is r, 2 is mu.

Name: Attribute number

Code: attributeNumber

Visibility condition: yes Default: — 

Parameter for the xml and html algorithms.

Name: Attribute name

Code: attributeName

Visibility condition: yes

Default: — 

Parameter for the xml and html algorithms.

Name: Result into variable Code: resultVariable Visibility condition: no

Default: — 

Variable for storing the result of the operation.

Name: More requests

Code: operations

Visibility condition: no

Default: — 

List of sequentially executed additional queries and assignments of results to a variable. Each value in the table represents a search query (analogous to the query field).

If the field is filled in, then the main request is executed first, and then all additional requests are executed with the same parameters.

The result of each is assigned to its corresponding variable in the table.

If any request fails, then:

  • The component will exit on a failure branch.

  • The error variable will be assigned the error text, supplemented by a prefix with the operation number, for example "Operation 3. Error text". The main operation corresponds to index 0.

  • Variables of previous executed queries will be filled, but subsequent queries will not.

Name: Abort on errors

Code: abortOnError

Visibility condition: yes

Default: yes

When performing multiple operations, the default error interrupts the process. The parameter allows you to continue performing the following operations, accumulating error information in a variable.

If at least one error occurs, exit is carried out via the Error branch, if it is specified.

If a basic operation that is specified separately by the 'query' property and not in the list of multiple operations fails, the process is always aborted.

Name: Error in variable

Code: errorVariable

Visibility condition: no Default: — 

Variable for storing the error.

Name: Transition

Code: transfer

Visibility condition: no

Default: — 

The component to which control is transferred upon successful completion of the operation.

Name: Transition, error Code: transferError Visibility condition: no

Default: — 

Component to which control is transferred upon error.

Examples

JSON algorithm

Review

  • When parsing an array of objects, you can specify a filter in (key:value); only objects with the specified key-value pair will be used for further parsing. A filter can only be based on one key-value pair. For example: "msgs"("msgSender":"ab"), "msgs" – contains an array of objects from which objects with the property "msgSender"="ab" will be selected.

  • Numbering of array elements starts from 0.

  • When specifying /DELIMITER=xxx/ at the beginning of the query string, the separator will be the specified characters (in this case, / cannot be a separator)

Example JSON document:
{
  "result": "ok",
  "errormsg": "",
  "info": {
    "https://some_url.domain/": {"/tmp/a/www": true}
  },
  "/x": [
    {
      "https://some_url.domain/": {"/tmp/a/www": ["first_elem", "sec_elem"]}
    }
  ],
  "chat": [
    {
      "sessId": "uvajoqnx0qcpbjoflxr",
      "msgs": [
        {
          "msgId": 8255,
          "msgDt": 1491292390,
          "msgData": {
            "type": "text",
            "data": "Good afternoon. Please choose the question you are interested in."
          },
          "msgSender": "op"
        },
        {
          "msgId": 8256,
          "msgDt": 1491292391,
          "msgData": {
            "type": "buttons",
            "data": "Contact user 6"
          },
          "msgSender": "ab"
        },
        {
          "msgId": 8257,
          "msgDt": 1491292392,
          "msgData": {
            "type": "buttons",
            "data": "Contact user 8"
          },
          "msgSender": "ab"
        }
      ]
    }
  ]
}
Table 3. Examples of JSON requests
Search Query Function Result

chat/0/sessId

Recovered content

uvajoqnx0qcpbjoflxr

Number of elements

1

chat/0/msgs(msgSender:ab)/1/msgId

Content

8257

Number of elements

1

chat/0/msgs(msgSender:ab)

Contents

 [
   {
     "msgId": 8256,
     "msgDt": 1491292391,
     "msgData": {
       "type": "buttons",
       "data": "Contact user 6"
     },
     "msgSender": "ab"
   },
   {
     "msgId": 8257,
     "msgDt": 1491292392,
     "msgData": {
       "type": "buttons",
       "data": "Contact user 8"
     },
     "msgSender": "ab"
   }
 ]

.

Number of elements

2

chat/0

List of keys

["sessId","msgs"]

/DELIMITER=@@@//x@@@0@@@https://some_url.domain/@@@/tmp/a/www@@@0

Content

\"first_elem\"

/DELIMITER=@/info@https://some_url.domain/@/tmp/a/www

Content

true

XML Algorithm

Example XML document:
<configuration version="16">
    <folder id="x.okteller.ru" label="x.okteller.ru" path="c:\rtx_mg3\Media\domains\x.okteller.ru" type="readwrite" rescanIntervalS="10" ignorePerms="false" autoNormalize="true">
        <device id="RXYYDPI-SVKNBMF-YADG7WD"></device>
        <minDiskFreePct>1</minDiskFreePct>
        <versioning></versioning>
        <copiers>0</copiers>
        <pullers>0</pullers>
        <hashers>0</hashers>
        <order>random</order>
    </folder>
    <folder id="okteller.ru" label="okteller.ru" path="c:\rtx_mg3\Media\domains\okteller.ru" type="readwrite" rescanIntervalS="10" ignorePerms="false" autoNormalize="true">
        <device id="RXYYDPI-SVKNBMF-YADG7WD"></device>
        <minDiskFreePct>1</minDiskFreePct>
        <versioning></versioning>
        <copiers>0</copiers>
        <pullers>0</pullers>
        <hashers>0</hashers>
        <order>random</order>
    </folder>
    <folder id="Common" label="Common" path="c:\rtx_mg3\Media\common" type="readwrite" rescanIntervalS="10" ignorePerms="false" autoNormalize="true">
        <device id="RXYYDPI-SVKNBMF-YADG7WD"></device>
        <minDiskFreePct>1</minDiskFreePct>
        <versioning></versioning>
        <copiers>0</copiers>
        <pullers>0</pullers>
        <hashers>0</hashers>
        <order>random</order>
    </folder>
    <device id="RXYYDPI-SVKNBMF-YADG7WD" name="Pavel" compression="metadata" introducer="false">
        <address>dynamic</address>
    </device>
    <gui enabled="true" tls="false" debugging="false">
        <address>127.0.0.1:8384</address>
        <apikey>jeopL9MghPvTmweKDeGcoXhwRtrdaVDP</apikey>
        <theme></theme>
    </gui>
    <options>
        <globalAnnounceEnabled>false</globalAnnounceEnabled>
        <localAnnounceEnabled>false</localAnnounceEnabled>
        <localAnnouncePort>21027</localAnnouncePort>
        <localAnnounceMCAddr></localAnnounceMCAddr>
        <maxSendKbps>0</maxSendKbps>
        <maxRecvKbps>0</maxRecvKbps>
        <reconnectionIntervalS>60</reconnectionIntervalS>
        <relaysEnabled>false</relaysEnabled>
        <relayReconnectIntervalM>10</relayReconnectIntervalM>
        <startBrowser>false</startBrowser>
        <natEnabled>false</natEnabled>
        <natLeaseMinutes>60</natLeaseMinutes>
        <natRenewalMinutes>30</natRenewalMinutes>
        <natTimeoutSeconds>10</natTimeoutSeconds>
    </options>
</configuration>
Table 4. XML Query Examples
Search Query Function Result

"configuration"/0/"folder"

Document

<folder id="Common" label="Common" path="c:\rtx_mg3\Media\common" type="readwrite" rescanIntervalS="10" ignorePerms="false" autoNormalize="true">
    <device id="RXYYDPI-SVKNBMF-YADG7WD"/>
    <minDiskFreePct>1</minDiskFreePct>
    <versioning/>
    <copiers>0</copiers>
    <pullers>0</pullers>
    <hashers>0</hashers>
    <order>random</order>
</folder>
<folder id="rootdomain.ru" label="rootdomain.ru" path="c:\rtx_mg3\Media\domains\rootdomain.ru" type="readwrite" rescanIntervalS="10" ignorePerms="false" autoNormalize="true">
    <device id="RXYYDPI-SVKNBMF-YADG7WD"/>
    <minDiskFreePct>1</minDiskFreePct>
    <versioning/>
    <copiers>0</copiers>
    <pullers>0</pullers>
    <hashers>0</hashers>
    <order>random</order>
</folder>
<folder id="x.rootdomain.ru" label="x.rootdomain.ru" path="c:\rtx_mg3\Media\domains\x.rootdomain.ru" type="readwrite" rescanIntervalS="10" ignorePerms="false" autoNormalize="true">
    <device id="RXYYDPI-SVKNBMF-YADG7WD"/>
    <minDiskFreePct>1</minDiskFreePct>
    <versioning/>
    <copiers>0</copiers>
    <pullers>0</pullers>
    <hashers>0</hashers>
    <order>random</order>
</folder>

.

"configuration"/0/"folder"("id":"x.rootdomain.ru")/0/"device"/0/

Document

<device id="RXYYDPI-SVKNBMF-YADG7WD"/>

.

"configuration"/0/"folder"("id":"x.rootdomain.ru")

Document

<folder id="x.rootdomain.ru" label="x.rootdomain.ru" path="c:\rtx_mg3\Media\domains\x.rootdomain.ru" type="readwrite" rescanIntervalS="10" ignorePerms="false" autoNormalize="true">
    <device id="RXYYDPI-SVKNBMF-YADG7WD"/>
    <minDiskFreePct>1</minDiskFreePct>
    <versioning/>
    <copiers>0</copiers>
    <pullers>0</pullers>
    <hashers>0</hashers>
    <order>random</order>
</folder>

.

Contents

<device id="RXYYDPI-SVKNBMF-YADG7WD"/>
<minDiskFreePct>1</minDiskFreePct>
<versioning/>
<copiers>0</copiers>
<pullers>0</pullers>
<hashers>0</hashers>
<order>random</order>

.

Attribute value.

Attribute name: "type"

readwrite

Attribute name.

Attribute number: 1

id

Number of attributes

7

HTML algorithm

Review

  • Search by tag name in double quotes, for example "br".

  • The sequence of tags is formed by the | separator, for example "html"|"head"|"title".

  • Getting one of the same tags is done by adding an index after the tag separator (starting from zero), for example "html"|"br"|1|"title".

  • By default, the transition to the search for the next tag occurs with the selection of the zero tag.

  • The "html"|0|"head"|0|"title"|0 query is equivalent to the "html"|"head"|"title" query, i.e. a query by tag name always returns a list.

  • The query can be built using indexes, for example "0|0|1".

  • Parentheses are used to refer to tag attributes.

  • The "html"|"head"|("charset") request will return a list of parent tags "head" that contain the "charset" attribute.

  • The "html"|"head"|("type="test/css") request will return a list of parent tags "head" in which the "type" attribute is equal to "test/css".

Example HTML document:
<HTML>
    <HEAD>
         <META a="1" b="2"/>
    </HEAD>
    <BODY>
        <p id="x1" class="abc">abc abc
        <a href="http://asdf.ru">link</a> <br>
        </p>
        <br><br>
        <p id="x2" class="abc" v="123">zxcv zxcv</p>
    </BODY>
</HTML>
Table 5. Examples of HTML requests
Search Query Function Result

("id"="x1")

Document

<p id="x1" class="abc">abc abc <a href="http://asdf.ru">link</a> <br></p>

.

Contents

abc abc <a href="http://asdf.ru">link</a> <br>

.

Number of elements

1

Attribute value. Attribute name: id

x1

"META"

Number of attributes

2

"p"

Document

<p id="x1" class="abc">abc abc <a href="http://asdf.ru">link</a> <br></p>

.

"p"|0

Document

<p id="x1" class="abc">abc abc <a href="http://asdf.ru">link</a> <br></p>

.

"p"|1

Document

<p id="x2" class="abc" v="123">zxcv zxcv</p>

.

"BODY"|"p"|("v")

Content

zxcv zxcv

See also