Wsrtjtyk

Suppose we have df1 that looks like this:

x1 = [{'partner': "Afghanistan", 'commodity': NaN}, 

      {'partner': "Zambia",      'commodity': 2}, 

      {'partner': "Germany",     'commodity': 2},

      {'partner': "Afghanistan", 'commodity': NaN},

      {'partner': "Canada",      'commodity': NaN},

      {'partner': "Italy",       'commodity': 3},

      {'partner': "Canada",      'commodity': NaN},

      {'partner': "USA",         'commodity': NaN}]



df1 = pd.DataFrame(x1)

What I want to do is see the list of values in partner that have the NaN value in commodity, but I don't want to have the same partner listed twice.

So my preferred result would look like this:

commodity_nan_partners=

Afghanistan

Canada

USA

and not:

Afghanistan

Afghanistan

Canada

Canada

USA

loc + isnull + drop_duplicates

You can filter your series and then drop duplicates:

res = df1.loc[df1['commodity'].isnull(), 'partner'].drop_duplicates()



print(res)



0    Afghanistan

4         Canada

7            USA

Name: partner, dtype: object

You can look for NaN values using isnull, then get unique values with unique or set:

>>> pd.Series(df1.loc[df1.commodity.isnull(),'partner'].unique())

0    Afghanistan

1         Canada

2            USA

dtype: object



# or

>>> pd.Series(list(set(df1.loc[df1.commodity.isnull(),'partner'])))

0         Canada

1    Afghanistan

2            USA

dtype: object

Step 1

Filter out to retain valid strings only:

v = df1.loc[df1.commodity.isna(), 'partner']

Or,

v = df1.partner[df1.commodity.isna()]

print(v)

0    Afghanistan

3    Afghanistan

4         Canada

6         Canada

7            USA

Name: partner, dtype: object

Step 2

Drop duplicates.

If you want a collection,

ingredients.unique()

array(['Afghanistan', 'Canada', 'USA'], dtype=object)

Or,

set(ingredients)

{'Afghanistan', 'Canada', 'USA'}

If you want a Series,

ser = ingredients.drop_duplicates().reset_index(drop=True)



0    Afghanistan

1         Canada

2            USA

Name: partner, dtype: object

If you want a DataFrame,

df = ser.to_frame()

May check with dropna , just provide a different Idea here .

set(df1.partner.tolist())-set(df1.dropna().partner.tolist())

Out[94]: {'Afghanistan', 'Canada', 'USA'}

Just another alternatives:

>>> df1[df1.isnull().any(axis=1)]['partner'].drop_duplicates()

0    Afghanistan

4         Canada

7            USA

Name: partner, dtype: object

Using loc + np.isnan

>>> df1.loc[np.isnan(df1.commodity), 'partner'].drop_duplicates()

0    Afghanistan

4         Canada

7            USA

Name: partner, dtype: object