How can I json-serialize a custom iterable?
up vote
4
down vote
favorite
I'd like to create a type that behaves as a named tuple except that it has a custom representation, which is also respected when serialized as JSON.
The naive by-the-books approach would be something like this:
from typing import NamedTuple
import json
class MyPair(NamedTuple):
left: str
right: str
def __repr__(self):
return self.left + ':' + self.right
class MyJSONEncoder(json.JSONEncoder):
def default(self, obj):
if isinstance(obj, MyPair):
return str(obj)
return json.JSONEncoder.default(self, obj)
Now print(MyPair('a', 'b')) will output a:b as intended, but print(json.dumps([MyPair('a', 'b')], cls=MyJSONEncoder)) will produce [["a", "b"]] because default() is only called if an object is not primitively serializable as JSON. Since my own type is a tuple, it will be serialized before I get a chance to intervene.
Is there any nice or not-so-nice way of achieving this without making MyPair not a Tuple or iterating over the entire document in a preprocessing step that replaces all MyPair objects by strings?
Edit: To address Joran's answer, I still want to retain the ability to serialize complex trees that just contain the occasional MyPairs. My minimal example might not have made that clear, sorry.
python json python-3.x types
add a comment |
up vote
4
down vote
favorite
I'd like to create a type that behaves as a named tuple except that it has a custom representation, which is also respected when serialized as JSON.
The naive by-the-books approach would be something like this:
from typing import NamedTuple
import json
class MyPair(NamedTuple):
left: str
right: str
def __repr__(self):
return self.left + ':' + self.right
class MyJSONEncoder(json.JSONEncoder):
def default(self, obj):
if isinstance(obj, MyPair):
return str(obj)
return json.JSONEncoder.default(self, obj)
Now print(MyPair('a', 'b')) will output a:b as intended, but print(json.dumps([MyPair('a', 'b')], cls=MyJSONEncoder)) will produce [["a", "b"]] because default() is only called if an object is not primitively serializable as JSON. Since my own type is a tuple, it will be serialized before I get a chance to intervene.
Is there any nice or not-so-nice way of achieving this without making MyPair not a Tuple or iterating over the entire document in a preprocessing step that replaces all MyPair objects by strings?
Edit: To address Joran's answer, I still want to retain the ability to serialize complex trees that just contain the occasional MyPairs. My minimal example might not have made that clear, sorry.
python json python-3.x types
Looks like you'd need to subclassJSONEncoderand override itsencodeand_make_iterencodemethods to special-case instances of your class.
– snakecharmerb
Nov 10 at 11:19
@snakecharmerb Yes, I had a look at those methods but they don't seem to be designed to be extended so I'd end up duplicating most of the functionality ofJSONEncoder:/
– Christian
Nov 12 at 14:37
add a comment |
up vote
4
down vote
favorite
up vote
4
down vote
favorite
I'd like to create a type that behaves as a named tuple except that it has a custom representation, which is also respected when serialized as JSON.
The naive by-the-books approach would be something like this:
from typing import NamedTuple
import json
class MyPair(NamedTuple):
left: str
right: str
def __repr__(self):
return self.left + ':' + self.right
class MyJSONEncoder(json.JSONEncoder):
def default(self, obj):
if isinstance(obj, MyPair):
return str(obj)
return json.JSONEncoder.default(self, obj)
Now print(MyPair('a', 'b')) will output a:b as intended, but print(json.dumps([MyPair('a', 'b')], cls=MyJSONEncoder)) will produce [["a", "b"]] because default() is only called if an object is not primitively serializable as JSON. Since my own type is a tuple, it will be serialized before I get a chance to intervene.
Is there any nice or not-so-nice way of achieving this without making MyPair not a Tuple or iterating over the entire document in a preprocessing step that replaces all MyPair objects by strings?
Edit: To address Joran's answer, I still want to retain the ability to serialize complex trees that just contain the occasional MyPairs. My minimal example might not have made that clear, sorry.
python json python-3.x types
I'd like to create a type that behaves as a named tuple except that it has a custom representation, which is also respected when serialized as JSON.
The naive by-the-books approach would be something like this:
from typing import NamedTuple
import json
class MyPair(NamedTuple):
left: str
right: str
def __repr__(self):
return self.left + ':' + self.right
class MyJSONEncoder(json.JSONEncoder):
def default(self, obj):
if isinstance(obj, MyPair):
return str(obj)
return json.JSONEncoder.default(self, obj)
Now print(MyPair('a', 'b')) will output a:b as intended, but print(json.dumps([MyPair('a', 'b')], cls=MyJSONEncoder)) will produce [["a", "b"]] because default() is only called if an object is not primitively serializable as JSON. Since my own type is a tuple, it will be serialized before I get a chance to intervene.
Is there any nice or not-so-nice way of achieving this without making MyPair not a Tuple or iterating over the entire document in a preprocessing step that replaces all MyPair objects by strings?
Edit: To address Joran's answer, I still want to retain the ability to serialize complex trees that just contain the occasional MyPairs. My minimal example might not have made that clear, sorry.
python json python-3.x types
python json python-3.x types
edited Nov 8 at 11:17
asked Nov 7 at 16:45
Christian
270218
270218
Looks like you'd need to subclassJSONEncoderand override itsencodeand_make_iterencodemethods to special-case instances of your class.
– snakecharmerb
Nov 10 at 11:19
@snakecharmerb Yes, I had a look at those methods but they don't seem to be designed to be extended so I'd end up duplicating most of the functionality ofJSONEncoder:/
– Christian
Nov 12 at 14:37
add a comment |
Looks like you'd need to subclassJSONEncoderand override itsencodeand_make_iterencodemethods to special-case instances of your class.
– snakecharmerb
Nov 10 at 11:19
@snakecharmerb Yes, I had a look at those methods but they don't seem to be designed to be extended so I'd end up duplicating most of the functionality ofJSONEncoder:/
– Christian
Nov 12 at 14:37
Looks like you'd need to subclass
JSONEncoder and override its encode and _make_iterencode methods to special-case instances of your class.– snakecharmerb
Nov 10 at 11:19
Looks like you'd need to subclass
JSONEncoder and override its encode and _make_iterencode methods to special-case instances of your class.– snakecharmerb
Nov 10 at 11:19
@snakecharmerb Yes, I had a look at those methods but they don't seem to be designed to be extended so I'd end up duplicating most of the functionality of
JSONEncoder :/– Christian
Nov 12 at 14:37
@snakecharmerb Yes, I had a look at those methods but they don't seem to be designed to be extended so I'd end up duplicating most of the functionality of
JSONEncoder :/– Christian
Nov 12 at 14:37
add a comment |
2 Answers
2
active
oldest
votes
up vote
2
down vote
just include the default parameter
def my_class_encoder(o):
if isinstance(o,MyClass):
return repr(o)
json.dumps(myClassInstance,default=my_class_encoder)
its easier to deal with than a real encoder
....
but really just add a def to your class
class MyPair(NamedTuple):
left: str
right: str
def serialize(self):
return list(self)
def __repr__(self):
return self.left + ':' + self.right
and then just
json.dumps(myClassInstance.serialize())
this has the benefit of being more clear in what its doing (at least imho)
Okay, my minimal example was probably to minimal. I of course still want to encode a proper JSON document that just includes MyPair objects somewhere in its tree.
– Christian
Nov 8 at 11:12
add a comment |
up vote
0
down vote
accepted
So I ended up reimplementing a JSONEncoder more or less from scratch. Since I don't need any fancy pretty-printing, this is fairly straightforward:
class MyJSONEncoder(json.JSONEncoder):
def __init__(self, *, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False,
indent=None, separators=None, default=None):
super().__init__(skipkeys=skipkeys, ensure_ascii=ensure_ascii, check_circular=check_circular,
allow_nan=allow_nan, sort_keys=sort_keys, indent=indent, separators=separators,
default=default)
self._serializers: Set[Tuple[Type, Callable]] = {
(MyPair, lambda pair: '"' + str(pair) + '"',)
}
def default(self, o):
return super().default(o)
def encode(self, o):
return ''.join(self.iterencode(o))
def iterencode(self, o, _one_shot=False):
for t, serializer in self._serializers:
if isinstance(o, t):
yield serializer(o)
break
else:
if isinstance(o, bool):
yield "true" if o else "false"
elif isinstance(o, str):
yield '"' + o + '"'
elif isinstance(o, bytes):
yield '"' + o.decode("utf-8") + '"'
elif isinstance(o, int) or isinstance(o, float) or isinstance(o, Decimal):
yield str(o)
elif isinstance(o, Dict):
yield '{'
for num, (key, value) in enumerate(o.items()):
yield bool(num) * ', ' + '"' + str(key) + '": '
yield from self.iterencode(value)
yield '}'
elif isinstance(o, Sequence):
yield '['
for num, value in enumerate(o):
yield bool(num) * ', '
yield from self.iterencode(value)
yield ']'
else:
yield self.default(o)
For custom types, adding the type name and the function that stringifies it to self._serializers and you should be good. The iterencode() behaves differently from the normal one (mainly in that it yields the brackets separately and not alongside the first or last element) but I couldn't see where this would break anything.
add a comment |
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
2
down vote
just include the default parameter
def my_class_encoder(o):
if isinstance(o,MyClass):
return repr(o)
json.dumps(myClassInstance,default=my_class_encoder)
its easier to deal with than a real encoder
....
but really just add a def to your class
class MyPair(NamedTuple):
left: str
right: str
def serialize(self):
return list(self)
def __repr__(self):
return self.left + ':' + self.right
and then just
json.dumps(myClassInstance.serialize())
this has the benefit of being more clear in what its doing (at least imho)
Okay, my minimal example was probably to minimal. I of course still want to encode a proper JSON document that just includes MyPair objects somewhere in its tree.
– Christian
Nov 8 at 11:12
add a comment |
up vote
2
down vote
just include the default parameter
def my_class_encoder(o):
if isinstance(o,MyClass):
return repr(o)
json.dumps(myClassInstance,default=my_class_encoder)
its easier to deal with than a real encoder
....
but really just add a def to your class
class MyPair(NamedTuple):
left: str
right: str
def serialize(self):
return list(self)
def __repr__(self):
return self.left + ':' + self.right
and then just
json.dumps(myClassInstance.serialize())
this has the benefit of being more clear in what its doing (at least imho)
Okay, my minimal example was probably to minimal. I of course still want to encode a proper JSON document that just includes MyPair objects somewhere in its tree.
– Christian
Nov 8 at 11:12
add a comment |
up vote
2
down vote
up vote
2
down vote
just include the default parameter
def my_class_encoder(o):
if isinstance(o,MyClass):
return repr(o)
json.dumps(myClassInstance,default=my_class_encoder)
its easier to deal with than a real encoder
....
but really just add a def to your class
class MyPair(NamedTuple):
left: str
right: str
def serialize(self):
return list(self)
def __repr__(self):
return self.left + ':' + self.right
and then just
json.dumps(myClassInstance.serialize())
this has the benefit of being more clear in what its doing (at least imho)
just include the default parameter
def my_class_encoder(o):
if isinstance(o,MyClass):
return repr(o)
json.dumps(myClassInstance,default=my_class_encoder)
its easier to deal with than a real encoder
....
but really just add a def to your class
class MyPair(NamedTuple):
left: str
right: str
def serialize(self):
return list(self)
def __repr__(self):
return self.left + ':' + self.right
and then just
json.dumps(myClassInstance.serialize())
this has the benefit of being more clear in what its doing (at least imho)
answered Nov 7 at 17:15
Joran Beasley
71.4k676115
71.4k676115
Okay, my minimal example was probably to minimal. I of course still want to encode a proper JSON document that just includes MyPair objects somewhere in its tree.
– Christian
Nov 8 at 11:12
add a comment |
Okay, my minimal example was probably to minimal. I of course still want to encode a proper JSON document that just includes MyPair objects somewhere in its tree.
– Christian
Nov 8 at 11:12
Okay, my minimal example was probably to minimal. I of course still want to encode a proper JSON document that just includes MyPair objects somewhere in its tree.
– Christian
Nov 8 at 11:12
Okay, my minimal example was probably to minimal. I of course still want to encode a proper JSON document that just includes MyPair objects somewhere in its tree.
– Christian
Nov 8 at 11:12
add a comment |
up vote
0
down vote
accepted
So I ended up reimplementing a JSONEncoder more or less from scratch. Since I don't need any fancy pretty-printing, this is fairly straightforward:
class MyJSONEncoder(json.JSONEncoder):
def __init__(self, *, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False,
indent=None, separators=None, default=None):
super().__init__(skipkeys=skipkeys, ensure_ascii=ensure_ascii, check_circular=check_circular,
allow_nan=allow_nan, sort_keys=sort_keys, indent=indent, separators=separators,
default=default)
self._serializers: Set[Tuple[Type, Callable]] = {
(MyPair, lambda pair: '"' + str(pair) + '"',)
}
def default(self, o):
return super().default(o)
def encode(self, o):
return ''.join(self.iterencode(o))
def iterencode(self, o, _one_shot=False):
for t, serializer in self._serializers:
if isinstance(o, t):
yield serializer(o)
break
else:
if isinstance(o, bool):
yield "true" if o else "false"
elif isinstance(o, str):
yield '"' + o + '"'
elif isinstance(o, bytes):
yield '"' + o.decode("utf-8") + '"'
elif isinstance(o, int) or isinstance(o, float) or isinstance(o, Decimal):
yield str(o)
elif isinstance(o, Dict):
yield '{'
for num, (key, value) in enumerate(o.items()):
yield bool(num) * ', ' + '"' + str(key) + '": '
yield from self.iterencode(value)
yield '}'
elif isinstance(o, Sequence):
yield '['
for num, value in enumerate(o):
yield bool(num) * ', '
yield from self.iterencode(value)
yield ']'
else:
yield self.default(o)
For custom types, adding the type name and the function that stringifies it to self._serializers and you should be good. The iterencode() behaves differently from the normal one (mainly in that it yields the brackets separately and not alongside the first or last element) but I couldn't see where this would break anything.
add a comment |
up vote
0
down vote
accepted
So I ended up reimplementing a JSONEncoder more or less from scratch. Since I don't need any fancy pretty-printing, this is fairly straightforward:
class MyJSONEncoder(json.JSONEncoder):
def __init__(self, *, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False,
indent=None, separators=None, default=None):
super().__init__(skipkeys=skipkeys, ensure_ascii=ensure_ascii, check_circular=check_circular,
allow_nan=allow_nan, sort_keys=sort_keys, indent=indent, separators=separators,
default=default)
self._serializers: Set[Tuple[Type, Callable]] = {
(MyPair, lambda pair: '"' + str(pair) + '"',)
}
def default(self, o):
return super().default(o)
def encode(self, o):
return ''.join(self.iterencode(o))
def iterencode(self, o, _one_shot=False):
for t, serializer in self._serializers:
if isinstance(o, t):
yield serializer(o)
break
else:
if isinstance(o, bool):
yield "true" if o else "false"
elif isinstance(o, str):
yield '"' + o + '"'
elif isinstance(o, bytes):
yield '"' + o.decode("utf-8") + '"'
elif isinstance(o, int) or isinstance(o, float) or isinstance(o, Decimal):
yield str(o)
elif isinstance(o, Dict):
yield '{'
for num, (key, value) in enumerate(o.items()):
yield bool(num) * ', ' + '"' + str(key) + '": '
yield from self.iterencode(value)
yield '}'
elif isinstance(o, Sequence):
yield '['
for num, value in enumerate(o):
yield bool(num) * ', '
yield from self.iterencode(value)
yield ']'
else:
yield self.default(o)
For custom types, adding the type name and the function that stringifies it to self._serializers and you should be good. The iterencode() behaves differently from the normal one (mainly in that it yields the brackets separately and not alongside the first or last element) but I couldn't see where this would break anything.
add a comment |
up vote
0
down vote
accepted
up vote
0
down vote
accepted
So I ended up reimplementing a JSONEncoder more or less from scratch. Since I don't need any fancy pretty-printing, this is fairly straightforward:
class MyJSONEncoder(json.JSONEncoder):
def __init__(self, *, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False,
indent=None, separators=None, default=None):
super().__init__(skipkeys=skipkeys, ensure_ascii=ensure_ascii, check_circular=check_circular,
allow_nan=allow_nan, sort_keys=sort_keys, indent=indent, separators=separators,
default=default)
self._serializers: Set[Tuple[Type, Callable]] = {
(MyPair, lambda pair: '"' + str(pair) + '"',)
}
def default(self, o):
return super().default(o)
def encode(self, o):
return ''.join(self.iterencode(o))
def iterencode(self, o, _one_shot=False):
for t, serializer in self._serializers:
if isinstance(o, t):
yield serializer(o)
break
else:
if isinstance(o, bool):
yield "true" if o else "false"
elif isinstance(o, str):
yield '"' + o + '"'
elif isinstance(o, bytes):
yield '"' + o.decode("utf-8") + '"'
elif isinstance(o, int) or isinstance(o, float) or isinstance(o, Decimal):
yield str(o)
elif isinstance(o, Dict):
yield '{'
for num, (key, value) in enumerate(o.items()):
yield bool(num) * ', ' + '"' + str(key) + '": '
yield from self.iterencode(value)
yield '}'
elif isinstance(o, Sequence):
yield '['
for num, value in enumerate(o):
yield bool(num) * ', '
yield from self.iterencode(value)
yield ']'
else:
yield self.default(o)
For custom types, adding the type name and the function that stringifies it to self._serializers and you should be good. The iterencode() behaves differently from the normal one (mainly in that it yields the brackets separately and not alongside the first or last element) but I couldn't see where this would break anything.
So I ended up reimplementing a JSONEncoder more or less from scratch. Since I don't need any fancy pretty-printing, this is fairly straightforward:
class MyJSONEncoder(json.JSONEncoder):
def __init__(self, *, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False,
indent=None, separators=None, default=None):
super().__init__(skipkeys=skipkeys, ensure_ascii=ensure_ascii, check_circular=check_circular,
allow_nan=allow_nan, sort_keys=sort_keys, indent=indent, separators=separators,
default=default)
self._serializers: Set[Tuple[Type, Callable]] = {
(MyPair, lambda pair: '"' + str(pair) + '"',)
}
def default(self, o):
return super().default(o)
def encode(self, o):
return ''.join(self.iterencode(o))
def iterencode(self, o, _one_shot=False):
for t, serializer in self._serializers:
if isinstance(o, t):
yield serializer(o)
break
else:
if isinstance(o, bool):
yield "true" if o else "false"
elif isinstance(o, str):
yield '"' + o + '"'
elif isinstance(o, bytes):
yield '"' + o.decode("utf-8") + '"'
elif isinstance(o, int) or isinstance(o, float) or isinstance(o, Decimal):
yield str(o)
elif isinstance(o, Dict):
yield '{'
for num, (key, value) in enumerate(o.items()):
yield bool(num) * ', ' + '"' + str(key) + '": '
yield from self.iterencode(value)
yield '}'
elif isinstance(o, Sequence):
yield '['
for num, value in enumerate(o):
yield bool(num) * ', '
yield from self.iterencode(value)
yield ']'
else:
yield self.default(o)
For custom types, adding the type name and the function that stringifies it to self._serializers and you should be good. The iterencode() behaves differently from the normal one (mainly in that it yields the brackets separately and not alongside the first or last element) but I couldn't see where this would break anything.
answered Nov 14 at 13:08
Christian
270218
270218
add a comment |
add a comment |
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53194044%2fhow-can-i-json-serialize-a-custom-iterable%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Looks like you'd need to subclass
JSONEncoderand override itsencodeand_make_iterencodemethods to special-case instances of your class.– snakecharmerb
Nov 10 at 11:19
@snakecharmerb Yes, I had a look at those methods but they don't seem to be designed to be extended so I'd end up duplicating most of the functionality of
JSONEncoder:/– Christian
Nov 12 at 14:37