Five Ways to Save a Python Dictionary to File

There are many ways to save a Python dictionary to file, and then load it into memory later. The five techniques I use most often are: pickle, json, numpy, string serialization, and custom function. Each technique has pros and cons.

Suppose you have a Python dictionary like so:

src_dict = {}
src_dict['name'] = 'Smith'
src_dict['age'] = 28
src_dict['height'] = 68.5
src_dict['salaried'] = True

1. Using the pickle module looks like:

import pickle

with open('saved_dict.pkl', 'wb') as f:
  pickle.dump(src_dict, f)
        
with open('saved_dict.pkl', 'rb') as f:
  loaded_dict = pickle.load(f)

v = loaded_dict['name']
print(v)
print(type(v))

2. Using the json module looks like:

import json

with open('saved_dict.json', 'w') as f:
  json.dump(src_dict, f)

with open('saved_dict.json') as f:
  loaded_dict = json.load(f)

v = loaded_dict['age']
print(v)
print(type(v))

3. Using the numpy module looks like:

import numpy as np

np.save('saved_dict.npy', src_dict) 

loaded_dict = np.load('saved_dict.npy', allow_pickle=True).item()

v = loaded_dict['height']
print(v)
print(type(v))

4. Using string serialization looks like:

with open('saved_dict.txt', 'w') as f:
  f.write(str(src_dict))

with open('saved_dict.txt', 'r') as f:
  loaded_dict = eval(f.read())

v = loaded_dict['age']
print(v)
print(type(v))

5. Using custom functions looks like:

def save_dict(dict, fn):
  with open(fn, 'w') as f:
    for (key,val) in dict.items():
      f.write(str(key) + ":" + str(val) + "\n")

def load_dict(fn):
  result = {}
  with open(fn, 'r') as f:
    for line in f:
      tokens = line.strip().split(":")
      result[tokens[0]] = tokens[1]
  return result

save_dict(src_dict, 'saved_dictary.txt')

loaded_dict = load_dict('saved_dictary.txt')
v = float(loaded_dict['height'])
print(v)
print(type(v))

The custom functions approach gives you maximum flexibility but requires more coding effort. The pickle, json, and numpy approaches are roughly equivalent — they incur an extra external dependency (bad) but are easy to use (good). The string serialization technique is a bit crude and I don’t use it unless it comes with existing code.



Three very different approaches for the cover art of “The Gods of Mars” (1913) by Edgar Rice Burroughs. Each style has artistic pros and cons for me. The book is the second in the Mars series, following “A Princess of Mars” (1912). Left: By artist Robert Abbett. Center: By artist Michael Whelan. Right: By artist Gino D’Achille.


Demo code:

# save_dict_techniques.py

import pickle
import json
import numpy as np

src_dict = {}
src_dict['name'] = 'Smith'
src_dict['age'] = 28
src_dict['height'] = 68.5
src_dict['salaried'] = True

print("\nSource dictionary: ")
print(src_dict)

# 1. pickle
print("\nSaving, loading using pickle, getting name: ")
with open('saved_dict.pkl', 'wb') as f:
  pickle.dump(src_dict, f)
        
with open('saved_dict.pkl', 'rb') as f:
  loaded_dict = pickle.load(f)

v = loaded_dict['name']
print(v)
print(type(v))

# 2. json
print("\nSaving, loading using josn, getting age: ")
with open('saved_dict.json', 'w') as f:
  json.dump(src_dict, f)

with open('saved_dict.json') as f:
  loaded_dict = json.load(f)

v = loaded_dict['age']
print(v)
print(type(v))

# 3. numpy
print("\nSaving, loading using numpy, getting height: ")
np.save('saved_dict.npy', src_dict) 

loaded_dict = np.load('saved_dict.npy', allow_pickle=True).item()

v = loaded_dict['height']
print(v)
print(type(v))

# 4. string serialization
print("\nSaving, loading using str serialization, getting age: ")
with open('saved_dict.txt', 'w') as f:
  f.write(str(src_dict))

with open('saved_dict.txt', 'r') as f:
  loaded_dict = eval(f.read())

v = loaded_dict['age']
print(v)
print(type(v))

# 5. custom function
print("\nSaving, loading using custom function, getting height: ")
def save_dict(dict, fn):
  with open(fn, 'w') as f:
    for (key,val) in dict.items():
      f.write(str(key) + ":" + str(val) + "\n")

def load_dict(fn):
  result = {}
  with open(fn, 'r') as f:
    for line in f:
      tokens = line.strip().split(":")
      result[tokens[0]] = tokens[1]
  return result

save_dict(src_dict, 'saved_dictary.txt')

loaded_dict = load_dict('saved_dictary.txt')
v = float(loaded_dict['height'])
print(v)
print(type(v))
This entry was posted in Machine Learning. Bookmark the permalink.