Receiving Data Header over TCP IP Node Red





.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box;
}







0















I am making a program to receive data from a 2D scanner that is sent through the data structure below.
I've done this same program in VBA but now that I migrate to the Node Red and I do not know how to get each data from the sensor data header on the Node Network.



Below is a photo of the sensor manual data header.



data header:
data header



enter image description here



enter image description here










share|improve this question































    0















    I am making a program to receive data from a 2D scanner that is sent through the data structure below.
    I've done this same program in VBA but now that I migrate to the Node Red and I do not know how to get each data from the sensor data header on the Node Network.



    Below is a photo of the sensor manual data header.



    data header:
    data header



    enter image description here



    enter image description here










    share|improve this question



























      0












      0








      0








      I am making a program to receive data from a 2D scanner that is sent through the data structure below.
      I've done this same program in VBA but now that I migrate to the Node Red and I do not know how to get each data from the sensor data header on the Node Network.



      Below is a photo of the sensor manual data header.



      data header:
      data header



      enter image description here



      enter image description here










      share|improve this question
















      I am making a program to receive data from a 2D scanner that is sent through the data structure below.
      I've done this same program in VBA but now that I migrate to the Node Red and I do not know how to get each data from the sensor data header on the Node Network.



      Below is a photo of the sensor manual data header.



      data header:
      data header



      enter image description here



      enter image description here







      tcp node-red






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Nov 24 '18 at 21:06







      Leandro Rodrigo de Oliveira

















      asked Nov 23 '18 at 16:41









      Leandro Rodrigo de OliveiraLeandro Rodrigo de Oliveira

      62




      62
























          1 Answer
          1






          active

          oldest

          votes


















          0














          So, basically your question is how to parse/unpack the binary data coming from the sensor to retrieve meaningful information. As it is normal in Node-RED there are two possible paths to follow: Do it by hand OR use some existing library to help you.



          Doing it by hand would require things like: store the binary data into a buffer, using function nodes in Node-RED to slice the buffer into separate pieces (using the protocol structure as your map), eventually escaping characters (if required by the protocol), calculating check-sums, etc.. It is not possible to help you further given the scarce amount of information provided in your post. However you certainly will find a supportive community in Node-RED forum, so I encourage you to go there for help.



          Another way is by taking advantage of existing libraries. This may slow you down as you need to research libraries, chose one and learn how to use it. I can try to help by providing some links for your reading:



          1- Node-RED node has a node to handle binary data (node-red-contrib-binary). I never used it so I can not tell you if it will offer all the capabilities you need. In general using pre-existing nodes (from Node-RED library) is the faster and safer way to proceed.
          https://flows.nodered.org/node/node-red-contrib-binary



          2- Additional libraries that could eventually be considered:



          https://github.com/bigeasy/packet



          https://github.com/substack/node-binary



          https://github.com/jDataView/jBinary



          https://www.npmjs.com/package/binary-parser-encoder



          https://www.npmjs.com/package/protobufjs



          Edit: Let´s say you chose to use the npm module binary-parser-encoder. The core of your flow will be a single function node with this code (tested ok with random data):



          var Packet = global.get('binary_parser');
          var buf = msg.payload;

          var typea = new Packet()
          .endianess("little")
          .uint16("magic")
          .uint16("packet_type")
          .uint32("packet_size")
          .uint16("header_size")
          .uint16("scan_number")
          .uint16("packet_number")
          .double("timestamp_raw")
          .double("timestamp_syncw")
          .uint32("status_flags")
          .uint32("scan_frequencye")
          .uint16("num_points_scan")
          .uint16("num_points_packet")
          .uint16("first_index")
          .uint32("first_angle")
          .uint32("angular_increment")
          .uint32("iq_input")
          .uint32("iq_overload")
          .double("iq_timestamp_raw")
          .double("iq_timestamp_sync")
          .uint8("header_padding");

          msg.payload = typea.parse(buf);
          return msg;





          share|improve this answer


























          • Thank you for your help. What you are looking for and how can I recommend and see if I can analyze the information. the msg.payload of the tcp request sends the information as shown below. Also follows a print of how I assembled the flow structure. If you have one more tip it will be very welcome.

            – Leandro Rodrigo de Oliveira
            Nov 24 '18 at 20:57











          • I just edited the answer to show you how the function node looks like. If you have issues on how to use the npm module in Node-RED please check the Node-RED forum as there are plenty of posts explaining how to do it. If the binary packet has always a fixed structure (which is apparently the case) then your flow will not be much complex.

            – AIOT MAKER
            Nov 24 '18 at 22:52











          • thank you very much for the help, I managed to advance my project and also learn more about the node red. I will continue requesting help in the node red community. For coinscidence I found an article dealing with the same sensor.

            – Leandro Rodrigo de Oliveira
            Nov 26 '18 at 10:48












          Your Answer






          StackExchange.ifUsing("editor", function () {
          StackExchange.using("externalEditor", function () {
          StackExchange.using("snippets", function () {
          StackExchange.snippets.init();
          });
          });
          }, "code-snippets");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "1"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53450301%2freceiving-data-header-over-tcp-ip-node-red%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          1 Answer
          1






          active

          oldest

          votes








          1 Answer
          1






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes









          0














          So, basically your question is how to parse/unpack the binary data coming from the sensor to retrieve meaningful information. As it is normal in Node-RED there are two possible paths to follow: Do it by hand OR use some existing library to help you.



          Doing it by hand would require things like: store the binary data into a buffer, using function nodes in Node-RED to slice the buffer into separate pieces (using the protocol structure as your map), eventually escaping characters (if required by the protocol), calculating check-sums, etc.. It is not possible to help you further given the scarce amount of information provided in your post. However you certainly will find a supportive community in Node-RED forum, so I encourage you to go there for help.



          Another way is by taking advantage of existing libraries. This may slow you down as you need to research libraries, chose one and learn how to use it. I can try to help by providing some links for your reading:



          1- Node-RED node has a node to handle binary data (node-red-contrib-binary). I never used it so I can not tell you if it will offer all the capabilities you need. In general using pre-existing nodes (from Node-RED library) is the faster and safer way to proceed.
          https://flows.nodered.org/node/node-red-contrib-binary



          2- Additional libraries that could eventually be considered:



          https://github.com/bigeasy/packet



          https://github.com/substack/node-binary



          https://github.com/jDataView/jBinary



          https://www.npmjs.com/package/binary-parser-encoder



          https://www.npmjs.com/package/protobufjs



          Edit: Let´s say you chose to use the npm module binary-parser-encoder. The core of your flow will be a single function node with this code (tested ok with random data):



          var Packet = global.get('binary_parser');
          var buf = msg.payload;

          var typea = new Packet()
          .endianess("little")
          .uint16("magic")
          .uint16("packet_type")
          .uint32("packet_size")
          .uint16("header_size")
          .uint16("scan_number")
          .uint16("packet_number")
          .double("timestamp_raw")
          .double("timestamp_syncw")
          .uint32("status_flags")
          .uint32("scan_frequencye")
          .uint16("num_points_scan")
          .uint16("num_points_packet")
          .uint16("first_index")
          .uint32("first_angle")
          .uint32("angular_increment")
          .uint32("iq_input")
          .uint32("iq_overload")
          .double("iq_timestamp_raw")
          .double("iq_timestamp_sync")
          .uint8("header_padding");

          msg.payload = typea.parse(buf);
          return msg;





          share|improve this answer


























          • Thank you for your help. What you are looking for and how can I recommend and see if I can analyze the information. the msg.payload of the tcp request sends the information as shown below. Also follows a print of how I assembled the flow structure. If you have one more tip it will be very welcome.

            – Leandro Rodrigo de Oliveira
            Nov 24 '18 at 20:57











          • I just edited the answer to show you how the function node looks like. If you have issues on how to use the npm module in Node-RED please check the Node-RED forum as there are plenty of posts explaining how to do it. If the binary packet has always a fixed structure (which is apparently the case) then your flow will not be much complex.

            – AIOT MAKER
            Nov 24 '18 at 22:52











          • thank you very much for the help, I managed to advance my project and also learn more about the node red. I will continue requesting help in the node red community. For coinscidence I found an article dealing with the same sensor.

            – Leandro Rodrigo de Oliveira
            Nov 26 '18 at 10:48
















          0














          So, basically your question is how to parse/unpack the binary data coming from the sensor to retrieve meaningful information. As it is normal in Node-RED there are two possible paths to follow: Do it by hand OR use some existing library to help you.



          Doing it by hand would require things like: store the binary data into a buffer, using function nodes in Node-RED to slice the buffer into separate pieces (using the protocol structure as your map), eventually escaping characters (if required by the protocol), calculating check-sums, etc.. It is not possible to help you further given the scarce amount of information provided in your post. However you certainly will find a supportive community in Node-RED forum, so I encourage you to go there for help.



          Another way is by taking advantage of existing libraries. This may slow you down as you need to research libraries, chose one and learn how to use it. I can try to help by providing some links for your reading:



          1- Node-RED node has a node to handle binary data (node-red-contrib-binary). I never used it so I can not tell you if it will offer all the capabilities you need. In general using pre-existing nodes (from Node-RED library) is the faster and safer way to proceed.
          https://flows.nodered.org/node/node-red-contrib-binary



          2- Additional libraries that could eventually be considered:



          https://github.com/bigeasy/packet



          https://github.com/substack/node-binary



          https://github.com/jDataView/jBinary



          https://www.npmjs.com/package/binary-parser-encoder



          https://www.npmjs.com/package/protobufjs



          Edit: Let´s say you chose to use the npm module binary-parser-encoder. The core of your flow will be a single function node with this code (tested ok with random data):



          var Packet = global.get('binary_parser');
          var buf = msg.payload;

          var typea = new Packet()
          .endianess("little")
          .uint16("magic")
          .uint16("packet_type")
          .uint32("packet_size")
          .uint16("header_size")
          .uint16("scan_number")
          .uint16("packet_number")
          .double("timestamp_raw")
          .double("timestamp_syncw")
          .uint32("status_flags")
          .uint32("scan_frequencye")
          .uint16("num_points_scan")
          .uint16("num_points_packet")
          .uint16("first_index")
          .uint32("first_angle")
          .uint32("angular_increment")
          .uint32("iq_input")
          .uint32("iq_overload")
          .double("iq_timestamp_raw")
          .double("iq_timestamp_sync")
          .uint8("header_padding");

          msg.payload = typea.parse(buf);
          return msg;





          share|improve this answer


























          • Thank you for your help. What you are looking for and how can I recommend and see if I can analyze the information. the msg.payload of the tcp request sends the information as shown below. Also follows a print of how I assembled the flow structure. If you have one more tip it will be very welcome.

            – Leandro Rodrigo de Oliveira
            Nov 24 '18 at 20:57











          • I just edited the answer to show you how the function node looks like. If you have issues on how to use the npm module in Node-RED please check the Node-RED forum as there are plenty of posts explaining how to do it. If the binary packet has always a fixed structure (which is apparently the case) then your flow will not be much complex.

            – AIOT MAKER
            Nov 24 '18 at 22:52











          • thank you very much for the help, I managed to advance my project and also learn more about the node red. I will continue requesting help in the node red community. For coinscidence I found an article dealing with the same sensor.

            – Leandro Rodrigo de Oliveira
            Nov 26 '18 at 10:48














          0












          0








          0







          So, basically your question is how to parse/unpack the binary data coming from the sensor to retrieve meaningful information. As it is normal in Node-RED there are two possible paths to follow: Do it by hand OR use some existing library to help you.



          Doing it by hand would require things like: store the binary data into a buffer, using function nodes in Node-RED to slice the buffer into separate pieces (using the protocol structure as your map), eventually escaping characters (if required by the protocol), calculating check-sums, etc.. It is not possible to help you further given the scarce amount of information provided in your post. However you certainly will find a supportive community in Node-RED forum, so I encourage you to go there for help.



          Another way is by taking advantage of existing libraries. This may slow you down as you need to research libraries, chose one and learn how to use it. I can try to help by providing some links for your reading:



          1- Node-RED node has a node to handle binary data (node-red-contrib-binary). I never used it so I can not tell you if it will offer all the capabilities you need. In general using pre-existing nodes (from Node-RED library) is the faster and safer way to proceed.
          https://flows.nodered.org/node/node-red-contrib-binary



          2- Additional libraries that could eventually be considered:



          https://github.com/bigeasy/packet



          https://github.com/substack/node-binary



          https://github.com/jDataView/jBinary



          https://www.npmjs.com/package/binary-parser-encoder



          https://www.npmjs.com/package/protobufjs



          Edit: Let´s say you chose to use the npm module binary-parser-encoder. The core of your flow will be a single function node with this code (tested ok with random data):



          var Packet = global.get('binary_parser');
          var buf = msg.payload;

          var typea = new Packet()
          .endianess("little")
          .uint16("magic")
          .uint16("packet_type")
          .uint32("packet_size")
          .uint16("header_size")
          .uint16("scan_number")
          .uint16("packet_number")
          .double("timestamp_raw")
          .double("timestamp_syncw")
          .uint32("status_flags")
          .uint32("scan_frequencye")
          .uint16("num_points_scan")
          .uint16("num_points_packet")
          .uint16("first_index")
          .uint32("first_angle")
          .uint32("angular_increment")
          .uint32("iq_input")
          .uint32("iq_overload")
          .double("iq_timestamp_raw")
          .double("iq_timestamp_sync")
          .uint8("header_padding");

          msg.payload = typea.parse(buf);
          return msg;





          share|improve this answer















          So, basically your question is how to parse/unpack the binary data coming from the sensor to retrieve meaningful information. As it is normal in Node-RED there are two possible paths to follow: Do it by hand OR use some existing library to help you.



          Doing it by hand would require things like: store the binary data into a buffer, using function nodes in Node-RED to slice the buffer into separate pieces (using the protocol structure as your map), eventually escaping characters (if required by the protocol), calculating check-sums, etc.. It is not possible to help you further given the scarce amount of information provided in your post. However you certainly will find a supportive community in Node-RED forum, so I encourage you to go there for help.



          Another way is by taking advantage of existing libraries. This may slow you down as you need to research libraries, chose one and learn how to use it. I can try to help by providing some links for your reading:



          1- Node-RED node has a node to handle binary data (node-red-contrib-binary). I never used it so I can not tell you if it will offer all the capabilities you need. In general using pre-existing nodes (from Node-RED library) is the faster and safer way to proceed.
          https://flows.nodered.org/node/node-red-contrib-binary



          2- Additional libraries that could eventually be considered:



          https://github.com/bigeasy/packet



          https://github.com/substack/node-binary



          https://github.com/jDataView/jBinary



          https://www.npmjs.com/package/binary-parser-encoder



          https://www.npmjs.com/package/protobufjs



          Edit: Let´s say you chose to use the npm module binary-parser-encoder. The core of your flow will be a single function node with this code (tested ok with random data):



          var Packet = global.get('binary_parser');
          var buf = msg.payload;

          var typea = new Packet()
          .endianess("little")
          .uint16("magic")
          .uint16("packet_type")
          .uint32("packet_size")
          .uint16("header_size")
          .uint16("scan_number")
          .uint16("packet_number")
          .double("timestamp_raw")
          .double("timestamp_syncw")
          .uint32("status_flags")
          .uint32("scan_frequencye")
          .uint16("num_points_scan")
          .uint16("num_points_packet")
          .uint16("first_index")
          .uint32("first_angle")
          .uint32("angular_increment")
          .uint32("iq_input")
          .uint32("iq_overload")
          .double("iq_timestamp_raw")
          .double("iq_timestamp_sync")
          .uint8("header_padding");

          msg.payload = typea.parse(buf);
          return msg;






          share|improve this answer














          share|improve this answer



          share|improve this answer








          edited Nov 24 '18 at 22:45

























          answered Nov 23 '18 at 23:38









          AIOT MAKERAIOT MAKER

          196126




          196126













          • Thank you for your help. What you are looking for and how can I recommend and see if I can analyze the information. the msg.payload of the tcp request sends the information as shown below. Also follows a print of how I assembled the flow structure. If you have one more tip it will be very welcome.

            – Leandro Rodrigo de Oliveira
            Nov 24 '18 at 20:57











          • I just edited the answer to show you how the function node looks like. If you have issues on how to use the npm module in Node-RED please check the Node-RED forum as there are plenty of posts explaining how to do it. If the binary packet has always a fixed structure (which is apparently the case) then your flow will not be much complex.

            – AIOT MAKER
            Nov 24 '18 at 22:52











          • thank you very much for the help, I managed to advance my project and also learn more about the node red. I will continue requesting help in the node red community. For coinscidence I found an article dealing with the same sensor.

            – Leandro Rodrigo de Oliveira
            Nov 26 '18 at 10:48



















          • Thank you for your help. What you are looking for and how can I recommend and see if I can analyze the information. the msg.payload of the tcp request sends the information as shown below. Also follows a print of how I assembled the flow structure. If you have one more tip it will be very welcome.

            – Leandro Rodrigo de Oliveira
            Nov 24 '18 at 20:57











          • I just edited the answer to show you how the function node looks like. If you have issues on how to use the npm module in Node-RED please check the Node-RED forum as there are plenty of posts explaining how to do it. If the binary packet has always a fixed structure (which is apparently the case) then your flow will not be much complex.

            – AIOT MAKER
            Nov 24 '18 at 22:52











          • thank you very much for the help, I managed to advance my project and also learn more about the node red. I will continue requesting help in the node red community. For coinscidence I found an article dealing with the same sensor.

            – Leandro Rodrigo de Oliveira
            Nov 26 '18 at 10:48

















          Thank you for your help. What you are looking for and how can I recommend and see if I can analyze the information. the msg.payload of the tcp request sends the information as shown below. Also follows a print of how I assembled the flow structure. If you have one more tip it will be very welcome.

          – Leandro Rodrigo de Oliveira
          Nov 24 '18 at 20:57





          Thank you for your help. What you are looking for and how can I recommend and see if I can analyze the information. the msg.payload of the tcp request sends the information as shown below. Also follows a print of how I assembled the flow structure. If you have one more tip it will be very welcome.

          – Leandro Rodrigo de Oliveira
          Nov 24 '18 at 20:57













          I just edited the answer to show you how the function node looks like. If you have issues on how to use the npm module in Node-RED please check the Node-RED forum as there are plenty of posts explaining how to do it. If the binary packet has always a fixed structure (which is apparently the case) then your flow will not be much complex.

          – AIOT MAKER
          Nov 24 '18 at 22:52





          I just edited the answer to show you how the function node looks like. If you have issues on how to use the npm module in Node-RED please check the Node-RED forum as there are plenty of posts explaining how to do it. If the binary packet has always a fixed structure (which is apparently the case) then your flow will not be much complex.

          – AIOT MAKER
          Nov 24 '18 at 22:52













          thank you very much for the help, I managed to advance my project and also learn more about the node red. I will continue requesting help in the node red community. For coinscidence I found an article dealing with the same sensor.

          – Leandro Rodrigo de Oliveira
          Nov 26 '18 at 10:48





          thank you very much for the help, I managed to advance my project and also learn more about the node red. I will continue requesting help in the node red community. For coinscidence I found an article dealing with the same sensor.

          – Leandro Rodrigo de Oliveira
          Nov 26 '18 at 10:48




















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Stack Overflow!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53450301%2freceiving-data-header-over-tcp-ip-node-red%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          這個網誌中的熱門文章

          Tangent Lines Diagram Along Smooth Curve

          Yusuf al-Mu'taman ibn Hud

          Zucchini