Create a hash table for large data in python

This post introduces how to create a hash table in python.

Text files can have duplicates which will overwrite existing keys in your dictionary (the python name for a hash table). We can create a unique set of the keys, and then use a dictionary comprehension to populate the dictionary.

sample_file.txt:

a
b
c
c

 

Python code:

with open("sample_file.txt") as f:
  keys = set(line.strip() for line in f.readlines())
my_dict = {key: 1 for key in keys if key}

>>> my_dict
{'a': 1, 'b': 1, 'c': 1}