Reversing algebraic equation with bitwise-XOR











up vote
2
down vote

favorite












I'm trying to reverse an encryption scheme, but I seem to have fallen into a pit when it comes to reversal using algebra.



The encryption scheme is as follows for a single char (using registers and constants):



encrypted_char= (original_char XOR dl) + al
where:
eax = eax.previous * c1 +c2
edx = (eax >> c3)
eax.0 is a known seeded constant.


I want to solve this equation algebraically for original_char, but I'm running into a few problems, namely with order of operations for getting original char on it's own. Thinking about wraparound for negative numbers is also giving me a headache.



If anyone had any pointers for how to solve for the original_char, it would be appreciated. My first thoughts are to just subtract al and then xor with dl, but I'm starting to feel confused at this point.










share|improve this question


























    up vote
    2
    down vote

    favorite












    I'm trying to reverse an encryption scheme, but I seem to have fallen into a pit when it comes to reversal using algebra.



    The encryption scheme is as follows for a single char (using registers and constants):



    encrypted_char= (original_char XOR dl) + al
    where:
    eax = eax.previous * c1 +c2
    edx = (eax >> c3)
    eax.0 is a known seeded constant.


    I want to solve this equation algebraically for original_char, but I'm running into a few problems, namely with order of operations for getting original char on it's own. Thinking about wraparound for negative numbers is also giving me a headache.



    If anyone had any pointers for how to solve for the original_char, it would be appreciated. My first thoughts are to just subtract al and then xor with dl, but I'm starting to feel confused at this point.










    share|improve this question
























      up vote
      2
      down vote

      favorite









      up vote
      2
      down vote

      favorite











      I'm trying to reverse an encryption scheme, but I seem to have fallen into a pit when it comes to reversal using algebra.



      The encryption scheme is as follows for a single char (using registers and constants):



      encrypted_char= (original_char XOR dl) + al
      where:
      eax = eax.previous * c1 +c2
      edx = (eax >> c3)
      eax.0 is a known seeded constant.


      I want to solve this equation algebraically for original_char, but I'm running into a few problems, namely with order of operations for getting original char on it's own. Thinking about wraparound for negative numbers is also giving me a headache.



      If anyone had any pointers for how to solve for the original_char, it would be appreciated. My first thoughts are to just subtract al and then xor with dl, but I'm starting to feel confused at this point.










      share|improve this question













      I'm trying to reverse an encryption scheme, but I seem to have fallen into a pit when it comes to reversal using algebra.



      The encryption scheme is as follows for a single char (using registers and constants):



      encrypted_char= (original_char XOR dl) + al
      where:
      eax = eax.previous * c1 +c2
      edx = (eax >> c3)
      eax.0 is a known seeded constant.


      I want to solve this equation algebraically for original_char, but I'm running into a few problems, namely with order of operations for getting original char on it's own. Thinking about wraparound for negative numbers is also giving me a headache.



      If anyone had any pointers for how to solve for the original_char, it would be appreciated. My first thoughts are to just subtract al and then xor with dl, but I'm starting to feel confused at this point.







      assembly encryption reverse-engineering






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked Nov 7 at 21:51









      comp.sci.intern

      356




      356
























          1 Answer
          1






          active

          oldest

          votes

















          up vote
          1
          down vote













          I played with a toy example before posting and my answer is as follows:



          bitwise xor has the same precedence as multiplication, I just flip it over. I already knew that XOR was the inverse of XOR, but I thought I should state it here.
          The resulting formula is as follows:
          (encrypted_char - al) XOR dl = al
          What goes into the larger registers doesn't need to be toyed with to arrive at the correct solution.



          I will solve the wraparound using the modulus operation with the correct size for my variables.



          Using the above methods I was able to reverse the code.






          share|improve this answer





















            Your Answer






            StackExchange.ifUsing("editor", function () {
            StackExchange.using("externalEditor", function () {
            StackExchange.using("snippets", function () {
            StackExchange.snippets.init();
            });
            });
            }, "code-snippets");

            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "1"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            convertImagesToLinks: true,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: 10,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });














             

            draft saved


            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53198391%2freversing-algebraic-equation-with-bitwise-xor%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            1 Answer
            1






            active

            oldest

            votes








            1 Answer
            1






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes








            up vote
            1
            down vote













            I played with a toy example before posting and my answer is as follows:



            bitwise xor has the same precedence as multiplication, I just flip it over. I already knew that XOR was the inverse of XOR, but I thought I should state it here.
            The resulting formula is as follows:
            (encrypted_char - al) XOR dl = al
            What goes into the larger registers doesn't need to be toyed with to arrive at the correct solution.



            I will solve the wraparound using the modulus operation with the correct size for my variables.



            Using the above methods I was able to reverse the code.






            share|improve this answer

























              up vote
              1
              down vote













              I played with a toy example before posting and my answer is as follows:



              bitwise xor has the same precedence as multiplication, I just flip it over. I already knew that XOR was the inverse of XOR, but I thought I should state it here.
              The resulting formula is as follows:
              (encrypted_char - al) XOR dl = al
              What goes into the larger registers doesn't need to be toyed with to arrive at the correct solution.



              I will solve the wraparound using the modulus operation with the correct size for my variables.



              Using the above methods I was able to reverse the code.






              share|improve this answer























                up vote
                1
                down vote










                up vote
                1
                down vote









                I played with a toy example before posting and my answer is as follows:



                bitwise xor has the same precedence as multiplication, I just flip it over. I already knew that XOR was the inverse of XOR, but I thought I should state it here.
                The resulting formula is as follows:
                (encrypted_char - al) XOR dl = al
                What goes into the larger registers doesn't need to be toyed with to arrive at the correct solution.



                I will solve the wraparound using the modulus operation with the correct size for my variables.



                Using the above methods I was able to reverse the code.






                share|improve this answer












                I played with a toy example before posting and my answer is as follows:



                bitwise xor has the same precedence as multiplication, I just flip it over. I already knew that XOR was the inverse of XOR, but I thought I should state it here.
                The resulting formula is as follows:
                (encrypted_char - al) XOR dl = al
                What goes into the larger registers doesn't need to be toyed with to arrive at the correct solution.



                I will solve the wraparound using the modulus operation with the correct size for my variables.



                Using the above methods I was able to reverse the code.







                share|improve this answer












                share|improve this answer



                share|improve this answer










                answered Nov 7 at 21:51









                comp.sci.intern

                356




                356






























                     

                    draft saved


                    draft discarded



















































                     


                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53198391%2freversing-algebraic-equation-with-bitwise-xor%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    這個網誌中的熱門文章

                    Post-Redirect-Get with Spring WebFlux and Thymeleaf

                    Xamarin.form Move up view when keyboard appear

                    JBPM : POST request for execute process go wrong