How to cause Windbg to download the compressed versions of MS public symbols
Microsoft state here:
The Microsoft Symbol Server provides compressed versions of the symbol
files. The files have an underscore at the end of the filename’s
extension to indicate that they are compressed. For example, the PDB
for ntdll.dll is available as ntdll.pd_.
I have 2 questions here:
The more general question: How can I cause Windbg to prefer the compressed version of the symbols? This would result in massive bandwidth savings. (Compressing c:Symbols on my computer resulted in a 68% reduction in size).
Sniffing the traffic reveals that the uncompressed version is tried first and then the compressed version (underscore at end of name).
and specific to MS Public Symbols: Are the compressed versions currently available at all? Trying to manually download a compressed versions of ntdll.pdb returns a 404 error.
>$ wget https://msdl.microsoft.com/download/symbols/ntdll.pdb/38A5841BD353770D9C800BF1AF6B17EB1/ntdll.pdb
...
ntdll.pdb 100%[=====================================================================================>] 1.46M 406KB/s in 4.4s
2018-11-11 01:16:56 (341 KB/s) - ‘ntdll.pdb’ saved [1534976/1534976]
>$ wget https://msdl.microsoft.com/download/symbols/ntdll.pdb/38A5841BD353770D9C800BF1AF6B17EB1/ntdll.pd_
....
HTTP request sent, awaiting response... 404 Not Found
2018-11-11 01:17:01 ERROR 404: Not Found.
Update:
I have now discovered that DbgHelp supports an option called SYMOPT_FAVOR_COMPRESSED which is explained as follows:
If there is both an uncompressed and a compressed file available, favor the compressed file. This option is good for slow connections.
The question now is how to enable this option in Windbg?
This option isn't documented in the officical Windbg documentation and setting it manually seems to only affect the UI level.
0:003> .symopt+ 0x800000
Symbol options are 0x830337:
0x00000001 - SYMOPT_CASE_INSENSITIVE
0x00000002 - SYMOPT_UNDNAME
0x00000004 - SYMOPT_DEFERRED_LOADS
0x00000010 - SYMOPT_LOAD_LINES
0x00000020 - SYMOPT_OMAP_FIND_NEAREST
0x00000100 - SYMOPT_NO_UNQUALIFIED_LOADS
0x00000200 - SYMOPT_FAIL_CRITICAL_ERRORS
0x00010000 - SYMOPT_AUTO_PUBLICS
0x00020000 - SYMOPT_NO_IMAGE_SEARCH
0x00800000 - SYMOPT_FAVOR_COMPRESSED
However inspecting the traffic using fiddler shows that the uncompressed version is still requested first.
debugging windbg debug-symbols
asked Nov 10 at 23:30
yossizahn
13
|
show 7 more comments
Microsoft state here:
The Microsoft Symbol Server provides compressed versions of the symbol
files. The files have an underscore at the end of the filename’s
extension to indicate that they are compressed. For example, the PDB
for ntdll.dll is available as ntdll.pd_.
I have 2 questions here:
The more general question: How can I cause Windbg to prefer the compressed version of the symbols? This would result in massive bandwidth savings. (Compressing c:Symbols on my computer resulted in a 68% reduction in size).
Sniffing the traffic reveals that the uncompressed version is tried first and then the compressed version (underscore at end of name).
and specific to MS Public Symbols: Are the compressed versions currently available at all? Trying to manually download a compressed versions of ntdll.pdb returns a 404 error.
>$ wget https://msdl.microsoft.com/download/symbols/ntdll.pdb/38A5841BD353770D9C800BF1AF6B17EB1/ntdll.pdb
...
ntdll.pdb 100%[=====================================================================================>] 1.46M 406KB/s in 4.4s
2018-11-11 01:16:56 (341 KB/s) - ‘ntdll.pdb’ saved [1534976/1534976]
>$ wget https://msdl.microsoft.com/download/symbols/ntdll.pdb/38A5841BD353770D9C800BF1AF6B17EB1/ntdll.pd_
....
HTTP request sent, awaiting response... 404 Not Found
2018-11-11 01:17:01 ERROR 404: Not Found.
Update:
I have now discovered that DbgHelp supports an option called SYMOPT_FAVOR_COMPRESSED which is explained as follows:
If there is both an uncompressed and a compressed file available, favor the compressed file. This option is good for slow connections.
The question now is how to enable this option in Windbg?
This option isn't documented in the officical Windbg documentation and setting it manually seems to only affect the UI level.
0:003> .symopt+ 0x800000
Symbol options are 0x830337:
0x00000001 - SYMOPT_CASE_INSENSITIVE
0x00000002 - SYMOPT_UNDNAME
0x00000004 - SYMOPT_DEFERRED_LOADS
0x00000010 - SYMOPT_LOAD_LINES
0x00000020 - SYMOPT_OMAP_FIND_NEAREST
0x00000100 - SYMOPT_NO_UNQUALIFIED_LOADS
0x00000200 - SYMOPT_FAIL_CRITICAL_ERRORS
0x00010000 - SYMOPT_AUTO_PUBLICS
0x00020000 - SYMOPT_NO_IMAGE_SEARCH
0x00800000 - SYMOPT_FAVOR_COMPRESSED
However inspecting the traffic using fiddler shows that the uncompressed version is still requested first.
debugging windbg debug-symbols
asked Nov 10 at 23:30
yossizahn
13
windbg downloads the compressed version only and it is expanded locally in your machine there is no bandwudth savings possible here use procmon or netmon to confirm the pdb directory is not browsable you need a curl .wget like app with proper useragent age and guid to retrieve them standalone
– blabb
Nov 11 at 7:40
Did you look at the wget session I posted? It shows that the decompressed version (no underscore - I actually opened the file to confirm that it is not compressed) is available with plain wget (no user-agent games needed) and the compressed version is unavailable.
– yossizahn
Nov 11 at 8:43
Which version of WinDbg and which version of dbghelp.dll and symsrv.dll do you use?
– Thomas Weller
Nov 12 at 14:15
FYI: compressing c:symbols with NTFS compression will save you ~68% disk space.
– Thomas Weller
Nov 12 at 14:20
@ThomasWeller I am using the latest version of Windbg from the Microsoft Store and it seems to be using DbgHelp version 10.0.18239.1000
– yossizahn
Nov 12 at 20:03
|
show 7 more comments
Microsoft state here:
The Microsoft Symbol Server provides compressed versions of the symbol
files. The files have an underscore at the end of the filename’s
extension to indicate that they are compressed. For example, the PDB
for ntdll.dll is available as ntdll.pd_.
I have 2 questions here:
The more general question: How can I cause Windbg to prefer the compressed version of the symbols? This would result in massive bandwidth savings. (Compressing c:Symbols on my computer resulted in a 68% reduction in size).
Sniffing the traffic reveals that the uncompressed version is tried first and then the compressed version (underscore at end of name).
and specific to MS Public Symbols: Are the compressed versions currently available at all? Trying to manually download a compressed versions of ntdll.pdb returns a 404 error.
>$ wget https://msdl.microsoft.com/download/symbols/ntdll.pdb/38A5841BD353770D9C800BF1AF6B17EB1/ntdll.pdb
...
ntdll.pdb 100%[=====================================================================================>] 1.46M 406KB/s in 4.4s
2018-11-11 01:16:56 (341 KB/s) - ‘ntdll.pdb’ saved [1534976/1534976]
>$ wget https://msdl.microsoft.com/download/symbols/ntdll.pdb/38A5841BD353770D9C800BF1AF6B17EB1/ntdll.pd_
....
HTTP request sent, awaiting response... 404 Not Found
2018-11-11 01:17:01 ERROR 404: Not Found.
Update:
I have now discovered that DbgHelp supports an option called SYMOPT_FAVOR_COMPRESSED which is explained as follows:
If there is both an uncompressed and a compressed file available, favor the compressed file. This option is good for slow connections.
The question now is how to enable this option in Windbg?
This option isn't documented in the officical Windbg documentation and setting it manually seems to only affect the UI level.
0:003> .symopt+ 0x800000
Symbol options are 0x830337:
0x00000001 - SYMOPT_CASE_INSENSITIVE
0x00000002 - SYMOPT_UNDNAME
0x00000004 - SYMOPT_DEFERRED_LOADS
0x00000010 - SYMOPT_LOAD_LINES
0x00000020 - SYMOPT_OMAP_FIND_NEAREST
0x00000100 - SYMOPT_NO_UNQUALIFIED_LOADS
0x00000200 - SYMOPT_FAIL_CRITICAL_ERRORS
0x00010000 - SYMOPT_AUTO_PUBLICS
0x00020000 - SYMOPT_NO_IMAGE_SEARCH
0x00800000 - SYMOPT_FAVOR_COMPRESSED
However inspecting the traffic using fiddler shows that the uncompressed version is still requested first.
debugging windbg debug-symbols
asked Nov 10 at 23:30
yossizahn
13
Microsoft state here:
The Microsoft Symbol Server provides compressed versions of the symbol
files. The files have an underscore at the end of the filename’s
extension to indicate that they are compressed. For example, the PDB
for ntdll.dll is available as ntdll.pd_.
I have 2 questions here:
The more general question: How can I cause Windbg to prefer the compressed version of the symbols? This would result in massive bandwidth savings. (Compressing c:Symbols on my computer resulted in a 68% reduction in size).
Sniffing the traffic reveals that the uncompressed version is tried first and then the compressed version (underscore at end of name).
and specific to MS Public Symbols: Are the compressed versions currently available at all? Trying to manually download a compressed versions of ntdll.pdb returns a 404 error.
>$ wget https://msdl.microsoft.com/download/symbols/ntdll.pdb/38A5841BD353770D9C800BF1AF6B17EB1/ntdll.pdb
...
ntdll.pdb 100%[=====================================================================================>] 1.46M 406KB/s in 4.4s
2018-11-11 01:16:56 (341 KB/s) - ‘ntdll.pdb’ saved [1534976/1534976]
>$ wget https://msdl.microsoft.com/download/symbols/ntdll.pdb/38A5841BD353770D9C800BF1AF6B17EB1/ntdll.pd_
....
HTTP request sent, awaiting response... 404 Not Found
2018-11-11 01:17:01 ERROR 404: Not Found.
Update:
I have now discovered that DbgHelp supports an option called SYMOPT_FAVOR_COMPRESSED which is explained as follows:
If there is both an uncompressed and a compressed file available, favor the compressed file. This option is good for slow connections.
The question now is how to enable this option in Windbg?
This option isn't documented in the officical Windbg documentation and setting it manually seems to only affect the UI level.
0:003> .symopt+ 0x800000
Symbol options are 0x830337:
0x00000001 - SYMOPT_CASE_INSENSITIVE
0x00000002 - SYMOPT_UNDNAME
0x00000004 - SYMOPT_DEFERRED_LOADS
0x00000010 - SYMOPT_LOAD_LINES
0x00000020 - SYMOPT_OMAP_FIND_NEAREST
0x00000100 - SYMOPT_NO_UNQUALIFIED_LOADS
0x00000200 - SYMOPT_FAIL_CRITICAL_ERRORS
0x00010000 - SYMOPT_AUTO_PUBLICS
0x00020000 - SYMOPT_NO_IMAGE_SEARCH
0x00800000 - SYMOPT_FAVOR_COMPRESSED
However inspecting the traffic using fiddler shows that the uncompressed version is still requested first.
debugging windbg debug-symbols
debugging windbg debug-symbols
asked Nov 10 at 23:30
yossizahn
13
asked Nov 10 at 23:30
yossizahn
13
edited Nov 12 at 20:05
asked Nov 10 at 23:30
yossizahn
13
asked Nov 10 at 23:30
yossizahn
13
asked Nov 10 at 23:30
yossizahn
13
13
windbg downloads the compressed version only and it is expanded locally in your machine there is no bandwudth savings possible here use procmon or netmon to confirm the pdb directory is not browsable you need a curl .wget like app with proper useragent age and guid to retrieve them standalone
– blabb
Nov 11 at 7:40
Did you look at the wget session I posted? It shows that the decompressed version (no underscore - I actually opened the file to confirm that it is not compressed) is available with plain wget (no user-agent games needed) and the compressed version is unavailable.
– yossizahn
Nov 11 at 8:43
Which version of WinDbg and which version of dbghelp.dll and symsrv.dll do you use?
– Thomas Weller
Nov 12 at 14:15
FYI: compressing c:symbols with NTFS compression will save you ~68% disk space.
– Thomas Weller
Nov 12 at 14:20
@ThomasWeller I am using the latest version of Windbg from the Microsoft Store and it seems to be using DbgHelp version 10.0.18239.1000
– yossizahn
Nov 12 at 20:03
|
show 7 more comments
windbg downloads the compressed version only and it is expanded locally in your machine there is no bandwudth savings possible here use procmon or netmon to confirm the pdb directory is not browsable you need a curl .wget like app with proper useragent age and guid to retrieve them standalone
– blabb
Nov 11 at 7:40
Did you look at the wget session I posted? It shows that the decompressed version (no underscore - I actually opened the file to confirm that it is not compressed) is available with plain wget (no user-agent games needed) and the compressed version is unavailable.
– yossizahn
Nov 11 at 8:43
Which version of WinDbg and which version of dbghelp.dll and symsrv.dll do you use?
– Thomas Weller
Nov 12 at 14:15
FYI: compressing c:symbols with NTFS compression will save you ~68% disk space.
– Thomas Weller
Nov 12 at 14:20
@ThomasWeller I am using the latest version of Windbg from the Microsoft Store and it seems to be using DbgHelp version 10.0.18239.1000
– yossizahn
Nov 12 at 20:03
windbg downloads the compressed version only and it is expanded locally in your machine there is no bandwudth savings possible here use procmon or netmon to confirm the pdb directory is not browsable you need a curl .wget like app with proper useragent age and guid to retrieve them standalone
– blabb
Nov 11 at 7:40
windbg downloads the compressed version only and it is expanded locally in your machine there is no bandwudth savings possible here use procmon or netmon to confirm the pdb directory is not browsable you need a curl .wget like app with proper useragent age and guid to retrieve them standalone
– blabb
Nov 11 at 7:40
Did you look at the wget session I posted? It shows that the decompressed version (no underscore - I actually opened the file to confirm that it is not compressed) is available with plain wget (no user-agent games needed) and the compressed version is unavailable.
– yossizahn
Nov 11 at 8:43
Did you look at the wget session I posted? It shows that the decompressed version (no underscore - I actually opened the file to confirm that it is not compressed) is available with plain wget (no user-agent games needed) and the compressed version is unavailable.
– yossizahn
Nov 11 at 8:43
Which version of WinDbg and which version of dbghelp.dll and symsrv.dll do you use?
– Thomas Weller
Nov 12 at 14:15
Which version of WinDbg and which version of dbghelp.dll and symsrv.dll do you use?
– Thomas Weller
Nov 12 at 14:15
FYI: compressing c:symbols with NTFS compression will save you ~68% disk space.
– Thomas Weller
Nov 12 at 14:20
FYI: compressing c:symbols with NTFS compression will save you ~68% disk space.
– Thomas Weller
Nov 12 at 14:20
@ThomasWeller I am using the latest version of Windbg from the Microsoft Store and it seems to be using DbgHelp version 10.0.18239.1000
– yossizahn
Nov 12 at 20:03
@ThomasWeller I am using the latest version of Windbg from the Microsoft Store and it seems to be using DbgHelp version 10.0.18239.1000
– yossizahn
Nov 12 at 20:03
|
show 7 more comments
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53244453%2fhow-to-cause-windbg-to-download-the-compressed-versions-of-ms-public-symbols%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53244453%2fhow-to-cause-windbg-to-download-the-compressed-versions-of-ms-public-symbols%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
windbg downloads the compressed version only and it is expanded locally in your machine there is no bandwudth savings possible here use procmon or netmon to confirm the pdb directory is not browsable you need a curl .wget like app with proper useragent age and guid to retrieve them standalone
– blabb
Nov 11 at 7:40
Did you look at the wget session I posted? It shows that the decompressed version (no underscore - I actually opened the file to confirm that it is not compressed) is available with plain wget (no user-agent games needed) and the compressed version is unavailable.
– yossizahn
Nov 11 at 8:43
Which version of WinDbg and which version of dbghelp.dll and symsrv.dll do you use?
– Thomas Weller
Nov 12 at 14:15
FYI: compressing c:symbols with NTFS compression will save you ~68% disk space.
– Thomas Weller
Nov 12 at 14:20
@ThomasWeller I am using the latest version of Windbg from the Microsoft Store and it seems to be using DbgHelp version 10.0.18239.1000
– yossizahn
Nov 12 at 20:03