While I was using TensorFlow’s imageNet trained model to extract the last pooling layer’s features as representation vectors for a new dataset of images, it worked just fine for around 21 – 22 images but then crashed with the following error:
File ".../lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2152, in _as_graph_def raise ValueError("GraphDef cannot be larger than 2GB.") ValueError: GraphDef cannot be larger than 2GB.
The reason caused the error: each call to
run_inference_on_image() adds nodes to the same TensorFlow graph, which eventually exceeds the maximum size (i.e., 2 GB).
The efficient solution:
run_inference_on_image() to run on multiple images. Call the instance of
sess.run() in a for loop that reads your image files. In this way, we will no longer need to reconstruct the entire model on each call, which will make processing each image much faster.
See the snippet below for some hints:
Rewrite the function run_inference_on_image()
I rename it to (and with one parameter) run_inference_on_multiple_images(path_to_your_image_files)
In the main function:
directory = os.path.dirname(os.getcwd()) + "/path-to-images/" run_inference_on_multiple_images(directory)
In the run_inference_on_multiple_images function
def run_inference_on_multiple_images(path_to_image_files): create_graph() ... for filename in os.listdir(path_to_image_files): if filename.endswith(".jpg"): image = (path_to_image_files+ filename) ... with tf.Session() as sess: ...
- overcome Graphdef cannot be larger than 2GB in tensorflow (pdf)