Wsrtjtyk

How can I use python list comprehension to replace values in one JSON file with linked values in a separate JSON file?

One will look like this and have an "a" value I need to use to replace a value in the other list, using "b" as the connector (a, b and c values are all unique ids):

{

   "records":[

      {

         "a": "7hk2k989u23lesdfsfd",

         "b":"b8",

      },

      {

         "a": "9ty562349u23lesdfsfd",

         "b":"b6",

      },

      {

         "a": "Ur233Fglesdfsfd",

         "b":"b2",

      }

   ]

}

the other will look like this where the "d"s need to be replaced with the corresponding "a" values where "b" is the key:

{

   "records":[

      {

         "c":00023414,

         "d":["b8","b6"]

      },

      {

         "c":0005814,

         "d":["b8","b2","b6"]

      }

   ]

}

So I end up with:

{

   "records":[

      {

         "c":00023414,

         "d":["7hk2k989u23lesdfsfd","9ty562349u23lesdfsfd"]

      },

      {

         "c":0005814,

         "d":["7hk2k989u23lesdfsfd","Ur233Fglesdfsfd","9ty562349u23lesdfsfd"]

      }

   ]

}

What's the right way to approach this using python, especially if I need code to be performant?

Your files aren't valid JSON. You should check with a JSON validator like JSON Lint

In [494]: import json



In [495]: with open('/Users/ado/Desktop/ab.json') as f:

     ...:     ab = json.load(f)

     ...:



In [496]: with open('/Users/ado/Desktop/cd.json') as f:

     ...:     cd = json.load(f)

     ...:

Notice that you could look at ab simply as a collection of related as and bs. This is a great time to use a dictionary mapping the bs to as

In [497]: d_ab = {r['b']: r['a'] for r in ab['records']}



In [498]: d_ab

Out[498]:

{'b2': 'Ur233Fglesdfsfd',

 'b6': '9ty562349u23lesdfsfd',

 'b8': '7hk2k989u23lesdfsfd'}

Now you can iterate over the records in cd and use a list comprehension to create the new values

In [499]: for r in cd['records']:

     ...:     r['d'] = [d_ab.get(d) for d in r['d']]

     ...:



In [500]: cd

Out[500]:

{'records': [{'c': 23414,

   'd': ['7hk2k989u23lesdfsfd', '9ty562349u23lesdfsfd']},

  {'c': 5814,

   'd': ['7hk2k989u23lesdfsfd', 'Ur233Fglesdfsfd', '9ty562349u23lesdfsfd']}]}

Finally, write the new contents to file

In [502]: with open('/Users/ado/Desktop/cd-mapped.json', 'w') as f:

     ...:     f.write(json.dumps(cd))

     ...:

This solution presupposes that in ab there is always as and bs in every record.

PS for funsies, you could use map and dict.get instead of the comprehension

In [505]: for r in cd['records']:

     ...:     r['d'] = list(map(d_ab.get, r['d']))

     ...:



In [506]: cd

Out[506]:

{'records': [{'c': 23414,

   'd': ['7hk2k989u23lesdfsfd', '9ty562349u23lesdfsfd']},

  {'c': 5814,

   'd': ['7hk2k989u23lesdfsfd', 'Ur233Fglesdfsfd', '9ty562349u23lesdfsfd']}]}

As far as performance goes, comprehensions often edge out maps

In [509]: %timeit for r in cd['records']: r['d'] = [d_ab.get(d) for d in r['d']]

     ...:

The slowest run took 7.19 times longer than the fastest. This could mean that an intermediate result is being cached.

1000000 loops, best of 3: 1.34 µs per loop



In [511]: %timeit for r in cd['records']: r['d'] = list(map(d_ab.get, r['d']))

The slowest run took 7.19 times longer than the fastest. This could mean that an intermediate result is being cached.

1000000 loops, best of 3: 1.74 µs per loop