update aTable set a,b,c = func(x,y,z,…)












0















I need a quick advice how-to. I mention that the following scenario is based on the use of c_api available already to my monetdblite compilation on 64bit, intention is to use it with some adhoc C written functions.



Short: how can I achieve or simulate the following scenario:
update aTable set a,b,c = func(x,y,z,…)



Long. Many algorithms are returning more than one variable as, for instance, multiple regression.



bool m_regression(IN const double **data, IN const int cols, IN const int rows, OUT double *fit_values, OUT double *residuals, OUT double *std_residuals, OUT double &p_value);


In order to minimize the transfer of data between monetdb and heavy computational function, all those results are generated in one step. Question is how can I transfer them back at once, minimizing computational time and memory traffic between monetdb and external C/C++(/R/Python) function?





My first thought to solve this is something like this:



1. update aTable set dummy = func_compute(x,y,z,…)



where dummy is a temporary __int64 field and func_compute will compute all the necessary outputs and store the result into a dummy pointer. To make sure is no issue with constant estimation, first returned value in the array will be the real dummy pointer, the rest just an incremented value of dummy + i;



2. update aTable set a = func_ret(dummy, 1), b= func_ret (dummy, 2), c= func_ret (dummy, 3) [, dummy=func_free(dummy)];



Assuming the func_ret will get the dummy in the same order that it was returned on first call, I would just copy the prepared result into provided storage; In case the order is not preserved, I will need an extra step to get the minimum (real dummy pointer), then to use the offset of current value to lookup in my array.



__int64 real_dummy = __inputs[0][0];

double *my_pointer_data = (double *) (real_dummy + __inputs[1][0] * sizeof(double)* row_count);

memcpy(__outputs[0], my_pointer_data, sizeof(double)* row_count);


// or ============================



__int64 real_dummy = minimum(__inputs[0]);

double *my_pointer_data = (double *) (real_dummy + __inputs[0][1] * sizeof(double)* row_count);

for (int i=0;i<row_count;i++)
__outputs[0][i] = my_pointer_data[__inputs[0][i] - real_dummy];


It is less relevant how am I going to free the temporary memory, can be in the last statement in update or in a new fake update statement using func_free.
Problem is that it doesn’t look to me that, even if I save some computational (big) time, the passing of the dummy is still done 3 times (any chance that memory is actually not copied?).



Is it any other better way of achieving this?










share|improve this question



























    0















    I need a quick advice how-to. I mention that the following scenario is based on the use of c_api available already to my monetdblite compilation on 64bit, intention is to use it with some adhoc C written functions.



    Short: how can I achieve or simulate the following scenario:
    update aTable set a,b,c = func(x,y,z,…)



    Long. Many algorithms are returning more than one variable as, for instance, multiple regression.



    bool m_regression(IN const double **data, IN const int cols, IN const int rows, OUT double *fit_values, OUT double *residuals, OUT double *std_residuals, OUT double &p_value);


    In order to minimize the transfer of data between monetdb and heavy computational function, all those results are generated in one step. Question is how can I transfer them back at once, minimizing computational time and memory traffic between monetdb and external C/C++(/R/Python) function?





    My first thought to solve this is something like this:



    1. update aTable set dummy = func_compute(x,y,z,…)



    where dummy is a temporary __int64 field and func_compute will compute all the necessary outputs and store the result into a dummy pointer. To make sure is no issue with constant estimation, first returned value in the array will be the real dummy pointer, the rest just an incremented value of dummy + i;



    2. update aTable set a = func_ret(dummy, 1), b= func_ret (dummy, 2), c= func_ret (dummy, 3) [, dummy=func_free(dummy)];



    Assuming the func_ret will get the dummy in the same order that it was returned on first call, I would just copy the prepared result into provided storage; In case the order is not preserved, I will need an extra step to get the minimum (real dummy pointer), then to use the offset of current value to lookup in my array.



    __int64 real_dummy = __inputs[0][0];

    double *my_pointer_data = (double *) (real_dummy + __inputs[1][0] * sizeof(double)* row_count);

    memcpy(__outputs[0], my_pointer_data, sizeof(double)* row_count);


    // or ============================



    __int64 real_dummy = minimum(__inputs[0]);

    double *my_pointer_data = (double *) (real_dummy + __inputs[0][1] * sizeof(double)* row_count);

    for (int i=0;i<row_count;i++)
    __outputs[0][i] = my_pointer_data[__inputs[0][i] - real_dummy];


    It is less relevant how am I going to free the temporary memory, can be in the last statement in update or in a new fake update statement using func_free.
    Problem is that it doesn’t look to me that, even if I save some computational (big) time, the passing of the dummy is still done 3 times (any chance that memory is actually not copied?).



    Is it any other better way of achieving this?










    share|improve this question

























      0












      0








      0








      I need a quick advice how-to. I mention that the following scenario is based on the use of c_api available already to my monetdblite compilation on 64bit, intention is to use it with some adhoc C written functions.



      Short: how can I achieve or simulate the following scenario:
      update aTable set a,b,c = func(x,y,z,…)



      Long. Many algorithms are returning more than one variable as, for instance, multiple regression.



      bool m_regression(IN const double **data, IN const int cols, IN const int rows, OUT double *fit_values, OUT double *residuals, OUT double *std_residuals, OUT double &p_value);


      In order to minimize the transfer of data between monetdb and heavy computational function, all those results are generated in one step. Question is how can I transfer them back at once, minimizing computational time and memory traffic between monetdb and external C/C++(/R/Python) function?





      My first thought to solve this is something like this:



      1. update aTable set dummy = func_compute(x,y,z,…)



      where dummy is a temporary __int64 field and func_compute will compute all the necessary outputs and store the result into a dummy pointer. To make sure is no issue with constant estimation, first returned value in the array will be the real dummy pointer, the rest just an incremented value of dummy + i;



      2. update aTable set a = func_ret(dummy, 1), b= func_ret (dummy, 2), c= func_ret (dummy, 3) [, dummy=func_free(dummy)];



      Assuming the func_ret will get the dummy in the same order that it was returned on first call, I would just copy the prepared result into provided storage; In case the order is not preserved, I will need an extra step to get the minimum (real dummy pointer), then to use the offset of current value to lookup in my array.



      __int64 real_dummy = __inputs[0][0];

      double *my_pointer_data = (double *) (real_dummy + __inputs[1][0] * sizeof(double)* row_count);

      memcpy(__outputs[0], my_pointer_data, sizeof(double)* row_count);


      // or ============================



      __int64 real_dummy = minimum(__inputs[0]);

      double *my_pointer_data = (double *) (real_dummy + __inputs[0][1] * sizeof(double)* row_count);

      for (int i=0;i<row_count;i++)
      __outputs[0][i] = my_pointer_data[__inputs[0][i] - real_dummy];


      It is less relevant how am I going to free the temporary memory, can be in the last statement in update or in a new fake update statement using func_free.
      Problem is that it doesn’t look to me that, even if I save some computational (big) time, the passing of the dummy is still done 3 times (any chance that memory is actually not copied?).



      Is it any other better way of achieving this?










      share|improve this question














      I need a quick advice how-to. I mention that the following scenario is based on the use of c_api available already to my monetdblite compilation on 64bit, intention is to use it with some adhoc C written functions.



      Short: how can I achieve or simulate the following scenario:
      update aTable set a,b,c = func(x,y,z,…)



      Long. Many algorithms are returning more than one variable as, for instance, multiple regression.



      bool m_regression(IN const double **data, IN const int cols, IN const int rows, OUT double *fit_values, OUT double *residuals, OUT double *std_residuals, OUT double &p_value);


      In order to minimize the transfer of data between monetdb and heavy computational function, all those results are generated in one step. Question is how can I transfer them back at once, minimizing computational time and memory traffic between monetdb and external C/C++(/R/Python) function?





      My first thought to solve this is something like this:



      1. update aTable set dummy = func_compute(x,y,z,…)



      where dummy is a temporary __int64 field and func_compute will compute all the necessary outputs and store the result into a dummy pointer. To make sure is no issue with constant estimation, first returned value in the array will be the real dummy pointer, the rest just an incremented value of dummy + i;



      2. update aTable set a = func_ret(dummy, 1), b= func_ret (dummy, 2), c= func_ret (dummy, 3) [, dummy=func_free(dummy)];



      Assuming the func_ret will get the dummy in the same order that it was returned on first call, I would just copy the prepared result into provided storage; In case the order is not preserved, I will need an extra step to get the minimum (real dummy pointer), then to use the offset of current value to lookup in my array.



      __int64 real_dummy = __inputs[0][0];

      double *my_pointer_data = (double *) (real_dummy + __inputs[1][0] * sizeof(double)* row_count);

      memcpy(__outputs[0], my_pointer_data, sizeof(double)* row_count);


      // or ============================



      __int64 real_dummy = minimum(__inputs[0]);

      double *my_pointer_data = (double *) (real_dummy + __inputs[0][1] * sizeof(double)* row_count);

      for (int i=0;i<row_count;i++)
      __outputs[0][i] = my_pointer_data[__inputs[0][i] - real_dummy];


      It is less relevant how am I going to free the temporary memory, can be in the last statement in update or in a new fake update statement using func_free.
      Problem is that it doesn’t look to me that, even if I save some computational (big) time, the passing of the dummy is still done 3 times (any chance that memory is actually not copied?).



      Is it any other better way of achieving this?







      c-api monetdblite






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked Nov 15 '18 at 11:45









      DanZDanZ

      53




      53
























          0






          active

          oldest

          votes











          Your Answer






          StackExchange.ifUsing("editor", function () {
          StackExchange.using("externalEditor", function () {
          StackExchange.using("snippets", function () {
          StackExchange.snippets.init();
          });
          });
          }, "code-snippets");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "1"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53318777%2fupdate-atable-set-a-b-c-funcx-y-z%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          0






          active

          oldest

          votes








          0






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes
















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Stack Overflow!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53318777%2fupdate-atable-set-a-b-c-funcx-y-z%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          這個網誌中的熱門文章

          Xamarin.form Move up view when keyboard appear

          Post-Redirect-Get with Spring WebFlux and Thymeleaf

          Anylogic : not able to use stopDelay()