Strange query results in Azure Cosmos DB











up vote
2
down vote

favorite
1












I have following documents in my Azure Cosmos DB:



{
"id": "token",
"User": {
"UserToken": "token",
"Email": "test@email.com"
},
"_ts": 1541493290
}


When I run the following query:



SELECT * FROM root  
WHERE ((root["User"]["UserToken"] = "token")
OR CONTAINS(root["User"]["Email"], "token"))
ORDER BY root["_ts"] DESC


Nothing is returned. But when I change it a bit. For example byconverting Email to email:



SELECT * FROM root  
WHERE ((root["User"]["UserToken"] = "token")
OR CONTAINS(root["User"]["email"], "token"))
ORDER BY root["_ts"] DESC


The result is found. Moreover when I remove ORDER BY clause, also query returns me a result. So the query is like following



SELECT * FROM root  
WHERE ((root["User"]["UserToken"] = "token")
OR CONTAINS(root["User"]["Email"], "token"))


Moreover, when I edit the document (like open it, add an empty line and save), some magic happens in the background and the document is found. For quite "new" documents (less than 1-3 months), I can search them without my "magic" trick.



Indexes definition is:



{
"indexingMode": "consistent",
"automatic": true,
"includedPaths": [
{
"path": "/*",
"indexes": [
{
"kind": "Range",
"dataType": "Number",
"precision": -1
},
{
"kind": "Hash",
"dataType": "String",
"precision": 3
}
]
}
],
"excludedPaths":
}


What I did wrong?



UPDATE the answer is not a full explanation but it helps a lot. Full explanation is in my blog (https://stapp.space/ridiculous-bug-in-azure-cosmos-db/)










share|improve this question




























    up vote
    2
    down vote

    favorite
    1












    I have following documents in my Azure Cosmos DB:



    {
    "id": "token",
    "User": {
    "UserToken": "token",
    "Email": "test@email.com"
    },
    "_ts": 1541493290
    }


    When I run the following query:



    SELECT * FROM root  
    WHERE ((root["User"]["UserToken"] = "token")
    OR CONTAINS(root["User"]["Email"], "token"))
    ORDER BY root["_ts"] DESC


    Nothing is returned. But when I change it a bit. For example byconverting Email to email:



    SELECT * FROM root  
    WHERE ((root["User"]["UserToken"] = "token")
    OR CONTAINS(root["User"]["email"], "token"))
    ORDER BY root["_ts"] DESC


    The result is found. Moreover when I remove ORDER BY clause, also query returns me a result. So the query is like following



    SELECT * FROM root  
    WHERE ((root["User"]["UserToken"] = "token")
    OR CONTAINS(root["User"]["Email"], "token"))


    Moreover, when I edit the document (like open it, add an empty line and save), some magic happens in the background and the document is found. For quite "new" documents (less than 1-3 months), I can search them without my "magic" trick.



    Indexes definition is:



    {
    "indexingMode": "consistent",
    "automatic": true,
    "includedPaths": [
    {
    "path": "/*",
    "indexes": [
    {
    "kind": "Range",
    "dataType": "Number",
    "precision": -1
    },
    {
    "kind": "Hash",
    "dataType": "String",
    "precision": 3
    }
    ]
    }
    ],
    "excludedPaths":
    }


    What I did wrong?



    UPDATE the answer is not a full explanation but it helps a lot. Full explanation is in my blog (https://stapp.space/ridiculous-bug-in-azure-cosmos-db/)










    share|improve this question


























      up vote
      2
      down vote

      favorite
      1









      up vote
      2
      down vote

      favorite
      1






      1





      I have following documents in my Azure Cosmos DB:



      {
      "id": "token",
      "User": {
      "UserToken": "token",
      "Email": "test@email.com"
      },
      "_ts": 1541493290
      }


      When I run the following query:



      SELECT * FROM root  
      WHERE ((root["User"]["UserToken"] = "token")
      OR CONTAINS(root["User"]["Email"], "token"))
      ORDER BY root["_ts"] DESC


      Nothing is returned. But when I change it a bit. For example byconverting Email to email:



      SELECT * FROM root  
      WHERE ((root["User"]["UserToken"] = "token")
      OR CONTAINS(root["User"]["email"], "token"))
      ORDER BY root["_ts"] DESC


      The result is found. Moreover when I remove ORDER BY clause, also query returns me a result. So the query is like following



      SELECT * FROM root  
      WHERE ((root["User"]["UserToken"] = "token")
      OR CONTAINS(root["User"]["Email"], "token"))


      Moreover, when I edit the document (like open it, add an empty line and save), some magic happens in the background and the document is found. For quite "new" documents (less than 1-3 months), I can search them without my "magic" trick.



      Indexes definition is:



      {
      "indexingMode": "consistent",
      "automatic": true,
      "includedPaths": [
      {
      "path": "/*",
      "indexes": [
      {
      "kind": "Range",
      "dataType": "Number",
      "precision": -1
      },
      {
      "kind": "Hash",
      "dataType": "String",
      "precision": 3
      }
      ]
      }
      ],
      "excludedPaths":
      }


      What I did wrong?



      UPDATE the answer is not a full explanation but it helps a lot. Full explanation is in my blog (https://stapp.space/ridiculous-bug-in-azure-cosmos-db/)










      share|improve this question















      I have following documents in my Azure Cosmos DB:



      {
      "id": "token",
      "User": {
      "UserToken": "token",
      "Email": "test@email.com"
      },
      "_ts": 1541493290
      }


      When I run the following query:



      SELECT * FROM root  
      WHERE ((root["User"]["UserToken"] = "token")
      OR CONTAINS(root["User"]["Email"], "token"))
      ORDER BY root["_ts"] DESC


      Nothing is returned. But when I change it a bit. For example byconverting Email to email:



      SELECT * FROM root  
      WHERE ((root["User"]["UserToken"] = "token")
      OR CONTAINS(root["User"]["email"], "token"))
      ORDER BY root["_ts"] DESC


      The result is found. Moreover when I remove ORDER BY clause, also query returns me a result. So the query is like following



      SELECT * FROM root  
      WHERE ((root["User"]["UserToken"] = "token")
      OR CONTAINS(root["User"]["Email"], "token"))


      Moreover, when I edit the document (like open it, add an empty line and save), some magic happens in the background and the document is found. For quite "new" documents (less than 1-3 months), I can search them without my "magic" trick.



      Indexes definition is:



      {
      "indexingMode": "consistent",
      "automatic": true,
      "includedPaths": [
      {
      "path": "/*",
      "indexes": [
      {
      "kind": "Range",
      "dataType": "Number",
      "precision": -1
      },
      {
      "kind": "Hash",
      "dataType": "String",
      "precision": 3
      }
      ]
      }
      ],
      "excludedPaths":
      }


      What I did wrong?



      UPDATE the answer is not a full explanation but it helps a lot. Full explanation is in my blog (https://stapp.space/ridiculous-bug-in-azure-cosmos-db/)







      azure azure-cosmosdb






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Nov 8 at 9:26

























      asked Nov 7 at 11:08









      Piotr Stapp

      13.8k44686




      13.8k44686
























          1 Answer
          1






          active

          oldest

          votes

















          up vote
          1
          down vote



          accepted










          CONTAINS(root["User"]["Email"], "token") won't work if you have strings indexed as Hash. They need to be Range with -1 precision. Hash only works for equality checks.



          That's why the lowercase one is working. Because it cannot find the property and it just ignores it, falling back to the equality check. The first one finds it, sees that it's not indexed as Range and it just fails to return.



          Changing indexing to this, will work:



          {
          "indexingMode": "consistent",
          "automatic": true,
          "includedPaths": [
          {
          "path": "/*",
          "indexes": [
          {
          "kind": "Range",
          "dataType": "Number",
          "precision": -1
          },
          {
          "kind": "Range",
          "dataType": "String",
          "precision": -1
          }
          ]
          }
          ],
          "excludedPaths":
          }


          On a side note, the _ts field is not the best way to do ordering based on creation. It is a unix timestamp in seconds, so any documents created in the same second won't be properly ordered.






          share|improve this answer























          • How to force "re-index"? I did above change, nothing happened.
            – Piotr Stapp
            Nov 7 at 13:23










          • And one more thing. The reason for the second is 100% understable. But still I don't know what is the difference between 1st and 3rd?
            – Piotr Stapp
            Nov 7 at 13:35










          • Re indexing will kick in automatically upon saving the changes. You can track the progress via code. However ordering with the _ts field might not work because it has its own indexing policy as it is a system defined meta property. I had many issues with it myself and I ended up making my own property for creation time with millisecond precision.
            – Nick Chapsas
            Nov 7 at 13:47













          Your Answer






          StackExchange.ifUsing("editor", function () {
          StackExchange.using("externalEditor", function () {
          StackExchange.using("snippets", function () {
          StackExchange.snippets.init();
          });
          });
          }, "code-snippets");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "1"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














           

          draft saved


          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53188291%2fstrange-query-results-in-azure-cosmos-db%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          1 Answer
          1






          active

          oldest

          votes








          1 Answer
          1






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes








          up vote
          1
          down vote



          accepted










          CONTAINS(root["User"]["Email"], "token") won't work if you have strings indexed as Hash. They need to be Range with -1 precision. Hash only works for equality checks.



          That's why the lowercase one is working. Because it cannot find the property and it just ignores it, falling back to the equality check. The first one finds it, sees that it's not indexed as Range and it just fails to return.



          Changing indexing to this, will work:



          {
          "indexingMode": "consistent",
          "automatic": true,
          "includedPaths": [
          {
          "path": "/*",
          "indexes": [
          {
          "kind": "Range",
          "dataType": "Number",
          "precision": -1
          },
          {
          "kind": "Range",
          "dataType": "String",
          "precision": -1
          }
          ]
          }
          ],
          "excludedPaths":
          }


          On a side note, the _ts field is not the best way to do ordering based on creation. It is a unix timestamp in seconds, so any documents created in the same second won't be properly ordered.






          share|improve this answer























          • How to force "re-index"? I did above change, nothing happened.
            – Piotr Stapp
            Nov 7 at 13:23










          • And one more thing. The reason for the second is 100% understable. But still I don't know what is the difference between 1st and 3rd?
            – Piotr Stapp
            Nov 7 at 13:35










          • Re indexing will kick in automatically upon saving the changes. You can track the progress via code. However ordering with the _ts field might not work because it has its own indexing policy as it is a system defined meta property. I had many issues with it myself and I ended up making my own property for creation time with millisecond precision.
            – Nick Chapsas
            Nov 7 at 13:47

















          up vote
          1
          down vote



          accepted










          CONTAINS(root["User"]["Email"], "token") won't work if you have strings indexed as Hash. They need to be Range with -1 precision. Hash only works for equality checks.



          That's why the lowercase one is working. Because it cannot find the property and it just ignores it, falling back to the equality check. The first one finds it, sees that it's not indexed as Range and it just fails to return.



          Changing indexing to this, will work:



          {
          "indexingMode": "consistent",
          "automatic": true,
          "includedPaths": [
          {
          "path": "/*",
          "indexes": [
          {
          "kind": "Range",
          "dataType": "Number",
          "precision": -1
          },
          {
          "kind": "Range",
          "dataType": "String",
          "precision": -1
          }
          ]
          }
          ],
          "excludedPaths":
          }


          On a side note, the _ts field is not the best way to do ordering based on creation. It is a unix timestamp in seconds, so any documents created in the same second won't be properly ordered.






          share|improve this answer























          • How to force "re-index"? I did above change, nothing happened.
            – Piotr Stapp
            Nov 7 at 13:23










          • And one more thing. The reason for the second is 100% understable. But still I don't know what is the difference between 1st and 3rd?
            – Piotr Stapp
            Nov 7 at 13:35










          • Re indexing will kick in automatically upon saving the changes. You can track the progress via code. However ordering with the _ts field might not work because it has its own indexing policy as it is a system defined meta property. I had many issues with it myself and I ended up making my own property for creation time with millisecond precision.
            – Nick Chapsas
            Nov 7 at 13:47















          up vote
          1
          down vote



          accepted







          up vote
          1
          down vote



          accepted






          CONTAINS(root["User"]["Email"], "token") won't work if you have strings indexed as Hash. They need to be Range with -1 precision. Hash only works for equality checks.



          That's why the lowercase one is working. Because it cannot find the property and it just ignores it, falling back to the equality check. The first one finds it, sees that it's not indexed as Range and it just fails to return.



          Changing indexing to this, will work:



          {
          "indexingMode": "consistent",
          "automatic": true,
          "includedPaths": [
          {
          "path": "/*",
          "indexes": [
          {
          "kind": "Range",
          "dataType": "Number",
          "precision": -1
          },
          {
          "kind": "Range",
          "dataType": "String",
          "precision": -1
          }
          ]
          }
          ],
          "excludedPaths":
          }


          On a side note, the _ts field is not the best way to do ordering based on creation. It is a unix timestamp in seconds, so any documents created in the same second won't be properly ordered.






          share|improve this answer














          CONTAINS(root["User"]["Email"], "token") won't work if you have strings indexed as Hash. They need to be Range with -1 precision. Hash only works for equality checks.



          That's why the lowercase one is working. Because it cannot find the property and it just ignores it, falling back to the equality check. The first one finds it, sees that it's not indexed as Range and it just fails to return.



          Changing indexing to this, will work:



          {
          "indexingMode": "consistent",
          "automatic": true,
          "includedPaths": [
          {
          "path": "/*",
          "indexes": [
          {
          "kind": "Range",
          "dataType": "Number",
          "precision": -1
          },
          {
          "kind": "Range",
          "dataType": "String",
          "precision": -1
          }
          ]
          }
          ],
          "excludedPaths":
          }


          On a side note, the _ts field is not the best way to do ordering based on creation. It is a unix timestamp in seconds, so any documents created in the same second won't be properly ordered.







          share|improve this answer














          share|improve this answer



          share|improve this answer








          edited Nov 7 at 11:20

























          answered Nov 7 at 11:14









          Nick Chapsas

          2,174213




          2,174213












          • How to force "re-index"? I did above change, nothing happened.
            – Piotr Stapp
            Nov 7 at 13:23










          • And one more thing. The reason for the second is 100% understable. But still I don't know what is the difference between 1st and 3rd?
            – Piotr Stapp
            Nov 7 at 13:35










          • Re indexing will kick in automatically upon saving the changes. You can track the progress via code. However ordering with the _ts field might not work because it has its own indexing policy as it is a system defined meta property. I had many issues with it myself and I ended up making my own property for creation time with millisecond precision.
            – Nick Chapsas
            Nov 7 at 13:47




















          • How to force "re-index"? I did above change, nothing happened.
            – Piotr Stapp
            Nov 7 at 13:23










          • And one more thing. The reason for the second is 100% understable. But still I don't know what is the difference between 1st and 3rd?
            – Piotr Stapp
            Nov 7 at 13:35










          • Re indexing will kick in automatically upon saving the changes. You can track the progress via code. However ordering with the _ts field might not work because it has its own indexing policy as it is a system defined meta property. I had many issues with it myself and I ended up making my own property for creation time with millisecond precision.
            – Nick Chapsas
            Nov 7 at 13:47


















          How to force "re-index"? I did above change, nothing happened.
          – Piotr Stapp
          Nov 7 at 13:23




          How to force "re-index"? I did above change, nothing happened.
          – Piotr Stapp
          Nov 7 at 13:23












          And one more thing. The reason for the second is 100% understable. But still I don't know what is the difference between 1st and 3rd?
          – Piotr Stapp
          Nov 7 at 13:35




          And one more thing. The reason for the second is 100% understable. But still I don't know what is the difference between 1st and 3rd?
          – Piotr Stapp
          Nov 7 at 13:35












          Re indexing will kick in automatically upon saving the changes. You can track the progress via code. However ordering with the _ts field might not work because it has its own indexing policy as it is a system defined meta property. I had many issues with it myself and I ended up making my own property for creation time with millisecond precision.
          – Nick Chapsas
          Nov 7 at 13:47






          Re indexing will kick in automatically upon saving the changes. You can track the progress via code. However ordering with the _ts field might not work because it has its own indexing policy as it is a system defined meta property. I had many issues with it myself and I ended up making my own property for creation time with millisecond precision.
          – Nick Chapsas
          Nov 7 at 13:47




















           

          draft saved


          draft discarded



















































           


          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53188291%2fstrange-query-results-in-azure-cosmos-db%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          這個網誌中的熱門文章

          Academy of Television Arts & Sciences

          L'Équipe

          1995 France bombings