Parsing Binary files that contains BCD (Binary Coded Decimal) values using Python Numpy
I have a binary file that has some fields encoded as BCD (Binary Coded Decimal). Example as below.
14 75 26 58 87 7F (Raw bytes in hex format).
I am using (np.void, 6) to read and convert from binary file and below is the output I am getting.
b'x14x75x26x58x87x7F'
But I would like to get the output as '14752658877', without the fill character 'F' using numpy.
Below is the code:
with open (filename, "rb") as f:
while True:
chunk = f.read(chunksize)
if (chunk):
dt = np.dtype([('a','b'), ('b', '>i4'), ('c', 'S15'),('d', np.str, 7),
('e', 'S7'), ('f', np.void, 6)])
x = np.frombuffer (chunk, dtype=dt)
print (x)
else:
break
Also, the input file contains many fixed length binary records. What is the efficient way to convert and store it as ascii file using numpy.
python numpy
asked Nov 21 '18 at 9:28
Raj KBRaj KB
184
|
show 1 more comment
I have a binary file that has some fields encoded as BCD (Binary Coded Decimal). Example as below.
14 75 26 58 87 7F (Raw bytes in hex format).
I am using (np.void, 6) to read and convert from binary file and below is the output I am getting.
b'x14x75x26x58x87x7F'
But I would like to get the output as '14752658877', without the fill character 'F' using numpy.
Below is the code:
with open (filename, "rb") as f:
while True:
chunk = f.read(chunksize)
if (chunk):
dt = np.dtype([('a','b'), ('b', '>i4'), ('c', 'S15'),('d', np.str, 7),
('e', 'S7'), ('f', np.void, 6)])
x = np.frombuffer (chunk, dtype=dt)
print (x)
else:
break
Also, the input file contains many fixed length binary records. What is the efficient way to convert and store it as ascii file using numpy.
python numpy
asked Nov 21 '18 at 9:28
Raj KBRaj KB
184
1
Show the example code.
– Ricardo Branco
Nov 21 '18 at 10:42
below is the code:with open (filename, "rb") as f: while True: chunk = f.read(chunksize) if (chunk): dt = np.dtype([('a','b'), ('b', '>i4'), ('c', 'S15'),('d', np.str, 7), ('e', 'S7'), ('f', np.void, 6)]) x = np.frombuffer (chunk, dtype=dt) print (x) else: break
– Raj KB
Nov 21 '18 at 16:24
Please edit your original question adding the above code keeping all the formatting and indentation.
– Ricardo Branco
Nov 21 '18 at 18:10
Hi Ricardo, I have edited the original question to add the code.
– Raj KB
Nov 21 '18 at 21:06
1
F is not a fill character. It is part of the hex value.
– Matt Messersmith
Nov 21 '18 at 21:10
|
show 1 more comment
I have a binary file that has some fields encoded as BCD (Binary Coded Decimal). Example as below.
14 75 26 58 87 7F (Raw bytes in hex format).
I am using (np.void, 6) to read and convert from binary file and below is the output I am getting.
b'x14x75x26x58x87x7F'
But I would like to get the output as '14752658877', without the fill character 'F' using numpy.
Below is the code:
with open (filename, "rb") as f:
while True:
chunk = f.read(chunksize)
if (chunk):
dt = np.dtype([('a','b'), ('b', '>i4'), ('c', 'S15'),('d', np.str, 7),
('e', 'S7'), ('f', np.void, 6)])
x = np.frombuffer (chunk, dtype=dt)
print (x)
else:
break
Also, the input file contains many fixed length binary records. What is the efficient way to convert and store it as ascii file using numpy.
python numpy
asked Nov 21 '18 at 9:28
Raj KBRaj KB
184
I have a binary file that has some fields encoded as BCD (Binary Coded Decimal). Example as below.
14 75 26 58 87 7F (Raw bytes in hex format).
I am using (np.void, 6) to read and convert from binary file and below is the output I am getting.
b'x14x75x26x58x87x7F'
But I would like to get the output as '14752658877', without the fill character 'F' using numpy.
Below is the code:
with open (filename, "rb") as f:
while True:
chunk = f.read(chunksize)
if (chunk):
dt = np.dtype([('a','b'), ('b', '>i4'), ('c', 'S15'),('d', np.str, 7),
('e', 'S7'), ('f', np.void, 6)])
x = np.frombuffer (chunk, dtype=dt)
print (x)
else:
break
Also, the input file contains many fixed length binary records. What is the efficient way to convert and store it as ascii file using numpy.
python numpy
python numpy
asked Nov 21 '18 at 9:28
Raj KBRaj KB
184
asked Nov 21 '18 at 9:28
Raj KBRaj KB
184
edited Nov 21 '18 at 21:05
Raj KB
asked Nov 21 '18 at 9:28
Raj KBRaj KB
184
asked Nov 21 '18 at 9:28
Raj KBRaj KB
184
asked Nov 21 '18 at 9:28
Raj KBRaj KB
184
184
1
Show the example code.
– Ricardo Branco
Nov 21 '18 at 10:42
below is the code:with open (filename, "rb") as f: while True: chunk = f.read(chunksize) if (chunk): dt = np.dtype([('a','b'), ('b', '>i4'), ('c', 'S15'),('d', np.str, 7), ('e', 'S7'), ('f', np.void, 6)]) x = np.frombuffer (chunk, dtype=dt) print (x) else: break
– Raj KB
Nov 21 '18 at 16:24
Please edit your original question adding the above code keeping all the formatting and indentation.
– Ricardo Branco
Nov 21 '18 at 18:10
Hi Ricardo, I have edited the original question to add the code.
– Raj KB
Nov 21 '18 at 21:06
1
F is not a fill character. It is part of the hex value.
– Matt Messersmith
Nov 21 '18 at 21:10
|
show 1 more comment
1
Show the example code.
– Ricardo Branco
Nov 21 '18 at 10:42
below is the code:with open (filename, "rb") as f: while True: chunk = f.read(chunksize) if (chunk): dt = np.dtype([('a','b'), ('b', '>i4'), ('c', 'S15'),('d', np.str, 7), ('e', 'S7'), ('f', np.void, 6)]) x = np.frombuffer (chunk, dtype=dt) print (x) else: break
– Raj KB
Nov 21 '18 at 16:24
Please edit your original question adding the above code keeping all the formatting and indentation.
– Ricardo Branco
Nov 21 '18 at 18:10
Hi Ricardo, I have edited the original question to add the code.
– Raj KB
Nov 21 '18 at 21:06
1
F is not a fill character. It is part of the hex value.
– Matt Messersmith
Nov 21 '18 at 21:10
1
1
Show the example code.
– Ricardo Branco
Nov 21 '18 at 10:42
Show the example code.
– Ricardo Branco
Nov 21 '18 at 10:42
below is the code:with open (filename, "rb") as f: while True: chunk = f.read(chunksize) if (chunk): dt = np.dtype([('a','b'), ('b', '>i4'), ('c', 'S15'),('d', np.str, 7), ('e', 'S7'), ('f', np.void, 6)]) x = np.frombuffer (chunk, dtype=dt) print (x) else: break
– Raj KB
Nov 21 '18 at 16:24
below is the code:with open (filename, "rb") as f: while True: chunk = f.read(chunksize) if (chunk): dt = np.dtype([('a','b'), ('b', '>i4'), ('c', 'S15'),('d', np.str, 7), ('e', 'S7'), ('f', np.void, 6)]) x = np.frombuffer (chunk, dtype=dt) print (x) else: break
– Raj KB
Nov 21 '18 at 16:24
Please edit your original question adding the above code keeping all the formatting and indentation.
– Ricardo Branco
Nov 21 '18 at 18:10
Please edit your original question adding the above code keeping all the formatting and indentation.
– Ricardo Branco
Nov 21 '18 at 18:10
Hi Ricardo, I have edited the original question to add the code.
– Raj KB
Nov 21 '18 at 21:06
Hi Ricardo, I have edited the original question to add the code.
– Raj KB
Nov 21 '18 at 21:06
1
1
F is not a fill character. It is part of the hex value.
– Matt Messersmith
Nov 21 '18 at 21:10
F is not a fill character. It is part of the hex value.
– Matt Messersmith
Nov 21 '18 at 21:10
|
show 1 more comment
1 Answer
1
active
oldest
votes
I don't know if numpy can somehow accelerate this, but a specalized function can be quickly constructed:
fastDict = {16*(i//10)+(i%10):i for i in range(100)}
def bcdToInteger(bcd):
result = 0
while bcd and bcd[0] in fastDict:
result *= 100
result += fastDict[bcd[0]]
bcd = bcd[1:]
if bcd and bcd[0] & 0xf0 <= 0x90:
result *= 10
result += bcd[0]>>4
if bcd[0] & 0xf <= 9:
result *= 10
result += bcd[0] & 0x0f
return result
>>> print (bcdToInteger(b'x14x75x26x58x87x7F')) # your sequence
14752658877
>>> print (bcdToInteger(b'x12x34xA0')) # first invalid nibble ends
1234
>>> print (bcdToInteger(b'x00x00x99')) # and so does an end of string
99
>>> print (bcdToInteger(b'x1F')) # a single nibble value
1
As long as you keep feeding it valid BCD bytes, it multiplies the result by 100 and adds the two new digits. Only the final byte needs some further inspection: if the highest nibble is valid, the result thus far gets multiplied by 10 and that nibble gets added. If the lowest nibble is valid as well, this is repeated.
The fastDict is to speed things up. It's a dictionary that returns the correct value for all 100 hex bytes from 00 to 99 so the number of actual calculations is as small as possible. You can do without the dictionary, but that means you have to do the comparisons and calculations in the if block for every single byte.
answered Nov 21 '18 at 22:24
usr2564301usr2564301
17.8k73372
Thanks for the solution. I am using binascii.hexlify(bcdvalue).decode('utf-8').rstrip('f') to get the preferred result. But I am looking for highly efficient solution as I have many such columns. Our daily record volumes are nearly 1 Billion records.
– Raj KB
Nov 22 '18 at 9:29
@RajKB: Well, my solution seems pretty efficient to me. As you can see in some other answers, these use an expensive bit-shift-and-compare twice per byte; and my code avoids that. Yet even faster code could be written using a custom extension in C, but I'm not going to attempt that.
– usr2564301
Nov 22 '18 at 10:40
Hi, I am trying your solution. But I got the below error while running the code. Can you please check. File "<ipython-input-9-6b5645ff319b>", line 7, in bcdToInteger if bcd and bcd[0] & 0xf0 <= 0x90: TypeError: unsupported operand type(s) for &: 'str' and 'int'
– Raj KB
Nov 22 '18 at 16:38
Please ignore the above comment.
– Raj KB
Nov 22 '18 at 16:46
Yes it did. Thanks alot for your answer.
– Raj KB
Nov 23 '18 at 10:16
|
show 1 more comment
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53408894%2fparsing-binary-files-that-contains-bcd-binary-coded-decimal-values-using-pytho%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
I don't know if numpy can somehow accelerate this, but a specalized function can be quickly constructed:
fastDict = {16*(i//10)+(i%10):i for i in range(100)}
def bcdToInteger(bcd):
result = 0
while bcd and bcd[0] in fastDict:
result *= 100
result += fastDict[bcd[0]]
bcd = bcd[1:]
if bcd and bcd[0] & 0xf0 <= 0x90:
result *= 10
result += bcd[0]>>4
if bcd[0] & 0xf <= 9:
result *= 10
result += bcd[0] & 0x0f
return result
>>> print (bcdToInteger(b'x14x75x26x58x87x7F')) # your sequence
14752658877
>>> print (bcdToInteger(b'x12x34xA0')) # first invalid nibble ends
1234
>>> print (bcdToInteger(b'x00x00x99')) # and so does an end of string
99
>>> print (bcdToInteger(b'x1F')) # a single nibble value
1
As long as you keep feeding it valid BCD bytes, it multiplies the result by 100 and adds the two new digits. Only the final byte needs some further inspection: if the highest nibble is valid, the result thus far gets multiplied by 10 and that nibble gets added. If the lowest nibble is valid as well, this is repeated.
The fastDict is to speed things up. It's a dictionary that returns the correct value for all 100 hex bytes from 00 to 99 so the number of actual calculations is as small as possible. You can do without the dictionary, but that means you have to do the comparisons and calculations in the if block for every single byte.
answered Nov 21 '18 at 22:24
usr2564301usr2564301
17.8k73372
Thanks for the solution. I am using binascii.hexlify(bcdvalue).decode('utf-8').rstrip('f') to get the preferred result. But I am looking for highly efficient solution as I have many such columns. Our daily record volumes are nearly 1 Billion records.
– Raj KB
Nov 22 '18 at 9:29
@RajKB: Well, my solution seems pretty efficient to me. As you can see in some other answers, these use an expensive bit-shift-and-compare twice per byte; and my code avoids that. Yet even faster code could be written using a custom extension in C, but I'm not going to attempt that.
– usr2564301
Nov 22 '18 at 10:40
Hi, I am trying your solution. But I got the below error while running the code. Can you please check. File "<ipython-input-9-6b5645ff319b>", line 7, in bcdToInteger if bcd and bcd[0] & 0xf0 <= 0x90: TypeError: unsupported operand type(s) for &: 'str' and 'int'
– Raj KB
Nov 22 '18 at 16:38
Please ignore the above comment.
– Raj KB
Nov 22 '18 at 16:46
Yes it did. Thanks alot for your answer.
– Raj KB
Nov 23 '18 at 10:16
|
show 1 more comment
I don't know if numpy can somehow accelerate this, but a specalized function can be quickly constructed:
fastDict = {16*(i//10)+(i%10):i for i in range(100)}
def bcdToInteger(bcd):
result = 0
while bcd and bcd[0] in fastDict:
result *= 100
result += fastDict[bcd[0]]
bcd = bcd[1:]
if bcd and bcd[0] & 0xf0 <= 0x90:
result *= 10
result += bcd[0]>>4
if bcd[0] & 0xf <= 9:
result *= 10
result += bcd[0] & 0x0f
return result
>>> print (bcdToInteger(b'x14x75x26x58x87x7F')) # your sequence
14752658877
>>> print (bcdToInteger(b'x12x34xA0')) # first invalid nibble ends
1234
>>> print (bcdToInteger(b'x00x00x99')) # and so does an end of string
99
>>> print (bcdToInteger(b'x1F')) # a single nibble value
1
As long as you keep feeding it valid BCD bytes, it multiplies the result by 100 and adds the two new digits. Only the final byte needs some further inspection: if the highest nibble is valid, the result thus far gets multiplied by 10 and that nibble gets added. If the lowest nibble is valid as well, this is repeated.
The fastDict is to speed things up. It's a dictionary that returns the correct value for all 100 hex bytes from 00 to 99 so the number of actual calculations is as small as possible. You can do without the dictionary, but that means you have to do the comparisons and calculations in the if block for every single byte.
answered Nov 21 '18 at 22:24
usr2564301usr2564301
17.8k73372
Thanks for the solution. I am using binascii.hexlify(bcdvalue).decode('utf-8').rstrip('f') to get the preferred result. But I am looking for highly efficient solution as I have many such columns. Our daily record volumes are nearly 1 Billion records.
– Raj KB
Nov 22 '18 at 9:29
@RajKB: Well, my solution seems pretty efficient to me. As you can see in some other answers, these use an expensive bit-shift-and-compare twice per byte; and my code avoids that. Yet even faster code could be written using a custom extension in C, but I'm not going to attempt that.
– usr2564301
Nov 22 '18 at 10:40
Hi, I am trying your solution. But I got the below error while running the code. Can you please check. File "<ipython-input-9-6b5645ff319b>", line 7, in bcdToInteger if bcd and bcd[0] & 0xf0 <= 0x90: TypeError: unsupported operand type(s) for &: 'str' and 'int'
– Raj KB
Nov 22 '18 at 16:38
Please ignore the above comment.
– Raj KB
Nov 22 '18 at 16:46
Yes it did. Thanks alot for your answer.
– Raj KB
Nov 23 '18 at 10:16
|
show 1 more comment
I don't know if numpy can somehow accelerate this, but a specalized function can be quickly constructed:
fastDict = {16*(i//10)+(i%10):i for i in range(100)}
def bcdToInteger(bcd):
result = 0
while bcd and bcd[0] in fastDict:
result *= 100
result += fastDict[bcd[0]]
bcd = bcd[1:]
if bcd and bcd[0] & 0xf0 <= 0x90:
result *= 10
result += bcd[0]>>4
if bcd[0] & 0xf <= 9:
result *= 10
result += bcd[0] & 0x0f
return result
>>> print (bcdToInteger(b'x14x75x26x58x87x7F')) # your sequence
14752658877
>>> print (bcdToInteger(b'x12x34xA0')) # first invalid nibble ends
1234
>>> print (bcdToInteger(b'x00x00x99')) # and so does an end of string
99
>>> print (bcdToInteger(b'x1F')) # a single nibble value
1
As long as you keep feeding it valid BCD bytes, it multiplies the result by 100 and adds the two new digits. Only the final byte needs some further inspection: if the highest nibble is valid, the result thus far gets multiplied by 10 and that nibble gets added. If the lowest nibble is valid as well, this is repeated.
The fastDict is to speed things up. It's a dictionary that returns the correct value for all 100 hex bytes from 00 to 99 so the number of actual calculations is as small as possible. You can do without the dictionary, but that means you have to do the comparisons and calculations in the if block for every single byte.
answered Nov 21 '18 at 22:24
usr2564301usr2564301
17.8k73372
I don't know if numpy can somehow accelerate this, but a specalized function can be quickly constructed:
fastDict = {16*(i//10)+(i%10):i for i in range(100)}
def bcdToInteger(bcd):
result = 0
while bcd and bcd[0] in fastDict:
result *= 100
result += fastDict[bcd[0]]
bcd = bcd[1:]
if bcd and bcd[0] & 0xf0 <= 0x90:
result *= 10
result += bcd[0]>>4
if bcd[0] & 0xf <= 9:
result *= 10
result += bcd[0] & 0x0f
return result
>>> print (bcdToInteger(b'x14x75x26x58x87x7F')) # your sequence
14752658877
>>> print (bcdToInteger(b'x12x34xA0')) # first invalid nibble ends
1234
>>> print (bcdToInteger(b'x00x00x99')) # and so does an end of string
99
>>> print (bcdToInteger(b'x1F')) # a single nibble value
1
As long as you keep feeding it valid BCD bytes, it multiplies the result by 100 and adds the two new digits. Only the final byte needs some further inspection: if the highest nibble is valid, the result thus far gets multiplied by 10 and that nibble gets added. If the lowest nibble is valid as well, this is repeated.
The fastDict is to speed things up. It's a dictionary that returns the correct value for all 100 hex bytes from 00 to 99 so the number of actual calculations is as small as possible. You can do without the dictionary, but that means you have to do the comparisons and calculations in the if block for every single byte.
answered Nov 21 '18 at 22:24
usr2564301usr2564301
17.8k73372
answered Nov 21 '18 at 22:24
usr2564301usr2564301
17.8k73372
answered Nov 21 '18 at 22:24
usr2564301usr2564301
17.8k73372
answered Nov 21 '18 at 22:24
usr2564301usr2564301
17.8k73372
17.8k73372
Thanks for the solution. I am using binascii.hexlify(bcdvalue).decode('utf-8').rstrip('f') to get the preferred result. But I am looking for highly efficient solution as I have many such columns. Our daily record volumes are nearly 1 Billion records.
– Raj KB
Nov 22 '18 at 9:29
@RajKB: Well, my solution seems pretty efficient to me. As you can see in some other answers, these use an expensive bit-shift-and-compare twice per byte; and my code avoids that. Yet even faster code could be written using a custom extension in C, but I'm not going to attempt that.
– usr2564301
Nov 22 '18 at 10:40
Hi, I am trying your solution. But I got the below error while running the code. Can you please check. File "<ipython-input-9-6b5645ff319b>", line 7, in bcdToInteger if bcd and bcd[0] & 0xf0 <= 0x90: TypeError: unsupported operand type(s) for &: 'str' and 'int'
– Raj KB
Nov 22 '18 at 16:38
Please ignore the above comment.
– Raj KB
Nov 22 '18 at 16:46
Yes it did. Thanks alot for your answer.
– Raj KB
Nov 23 '18 at 10:16
|
show 1 more comment
Thanks for the solution. I am using binascii.hexlify(bcdvalue).decode('utf-8').rstrip('f') to get the preferred result. But I am looking for highly efficient solution as I have many such columns. Our daily record volumes are nearly 1 Billion records.
– Raj KB
Nov 22 '18 at 9:29
@RajKB: Well, my solution seems pretty efficient to me. As you can see in some other answers, these use an expensive bit-shift-and-compare twice per byte; and my code avoids that. Yet even faster code could be written using a custom extension in C, but I'm not going to attempt that.
– usr2564301
Nov 22 '18 at 10:40
Hi, I am trying your solution. But I got the below error while running the code. Can you please check. File "<ipython-input-9-6b5645ff319b>", line 7, in bcdToInteger if bcd and bcd[0] & 0xf0 <= 0x90: TypeError: unsupported operand type(s) for &: 'str' and 'int'
– Raj KB
Nov 22 '18 at 16:38
Please ignore the above comment.
– Raj KB
Nov 22 '18 at 16:46
Yes it did. Thanks alot for your answer.
– Raj KB
Nov 23 '18 at 10:16
Thanks for the solution. I am using binascii.hexlify(bcdvalue).decode('utf-8').rstrip('f') to get the preferred result. But I am looking for highly efficient solution as I have many such columns. Our daily record volumes are nearly 1 Billion records.
– Raj KB
Nov 22 '18 at 9:29
Thanks for the solution. I am using binascii.hexlify(bcdvalue).decode('utf-8').rstrip('f') to get the preferred result. But I am looking for highly efficient solution as I have many such columns. Our daily record volumes are nearly 1 Billion records.
– Raj KB
Nov 22 '18 at 9:29
@RajKB: Well, my solution seems pretty efficient to me. As you can see in some other answers, these use an expensive bit-shift-and-compare twice per byte; and my code avoids that. Yet even faster code could be written using a custom extension in C, but I'm not going to attempt that.
– usr2564301
Nov 22 '18 at 10:40
@RajKB: Well, my solution seems pretty efficient to me. As you can see in some other answers, these use an expensive bit-shift-and-compare twice per byte; and my code avoids that. Yet even faster code could be written using a custom extension in C, but I'm not going to attempt that.
– usr2564301
Nov 22 '18 at 10:40
Hi, I am trying your solution. But I got the below error while running the code. Can you please check. File "<ipython-input-9-6b5645ff319b>", line 7, in bcdToInteger if bcd and bcd[0] & 0xf0 <= 0x90: TypeError: unsupported operand type(s) for &: 'str' and 'int'
– Raj KB
Nov 22 '18 at 16:38
Hi, I am trying your solution. But I got the below error while running the code. Can you please check. File "<ipython-input-9-6b5645ff319b>", line 7, in bcdToInteger if bcd and bcd[0] & 0xf0 <= 0x90: TypeError: unsupported operand type(s) for &: 'str' and 'int'
– Raj KB
Nov 22 '18 at 16:38
Please ignore the above comment.
– Raj KB
Nov 22 '18 at 16:46
Please ignore the above comment.
– Raj KB
Nov 22 '18 at 16:46
Yes it did. Thanks alot for your answer.
– Raj KB
Nov 23 '18 at 10:16
Yes it did. Thanks alot for your answer.
– Raj KB
Nov 23 '18 at 10:16
|
show 1 more comment
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53408894%2fparsing-binary-files-that-contains-bcd-binary-coded-decimal-values-using-pytho%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
Show the example code.
– Ricardo Branco
Nov 21 '18 at 10:42
below is the code:with open (filename, "rb") as f: while True: chunk = f.read(chunksize) if (chunk): dt = np.dtype([('a','b'), ('b', '>i4'), ('c', 'S15'),('d', np.str, 7), ('e', 'S7'), ('f', np.void, 6)]) x = np.frombuffer (chunk, dtype=dt) print (x) else: break
– Raj KB
Nov 21 '18 at 16:24
Please edit your original question adding the above code keeping all the formatting and indentation.
– Ricardo Branco
Nov 21 '18 at 18:10
Hi Ricardo, I have edited the original question to add the code.
– Raj KB
Nov 21 '18 at 21:06
1
F is not a fill character. It is part of the hex value.
– Matt Messersmith
Nov 21 '18 at 21:10