PDF to TIFF to TXT: Bash Script Automation

A few weeks ago I started into a project to take a scanned PDF image and turn it into text. At the time I was doing this manually by taking the PDF file and converting it to a TIFF (image file) and then running that through tesseract-OCR engine to output a TXT file. Then I would do some regular expressions on the file to pull out manual line breaks that the OCR process stuck into the file.

Since I was taking a class in Python at the time I thought I would do this using Python. But when I actually started coding the steps, I realized that a bash script would do everything I needed and I already knew how to do most of it. So that is what I ended up with.

I will walk you through what I understand about it. I did get help with the regular expression (line 12) and mostly understand it. But here is a stab at helping you see what is going on. And, I am certainly willing for anyone to offer suggestions to make the script better.

To use the script you invoke it with the name of the script and then the name of the PDF like this: pdf2tiff2txt.sh filename.pdf

  • Line 1: This says that the script is a bash script.
  • Lines 2-5: A failsafe to tell the user that they need to type in the command for the script (my script is called pdf2tiff2txt.sh) and then the location of the PDF that is to be converted.
  • Lines 6-9: A failsafe that if a non-existent file name is given then the user knows the file does not exist in the location they indicated.
  • Line 10: Converts the input file into a TIFF with a resolution of 300 dpi and an 8-bit color depth. It also outputs the original file name with a .tiff extension.
  • Line 11: Uses the tesseract OCR engine to convert the file from an image to a text file. The .txt file extension is automatically added by tesseract.
  • Line 12: The magical regex line. All carriage returns are converted to asterisks. Then double asterisks are converted to double beginning lines. Single asterisks are converted to spaces. The double beginning lines are converted to double carriage returns. Finally the new file is output as the original file name plus .converted.txt.
  • Lines 13 and 14: Deletes the intermediary files that were created. I don’t ever have need for these so I can safely delete them. This is the section where I think my script could be improved. I would guess there is a way to create the intermediary files as temporary files that get automatically deleted as soon as they are used.

I am currently using this in production and it does exactly what I need. I am very pleased with the output and want to thank my friend Brett for his help with the regex and the failsafe lines.

Use Tesseract OCR with PDF File

Goal — Copy Text from PDF Scan

If a PDF is created from a computer file then the text is embedded as part of the file. You can simply copy and paste the text from the PDF. But if the PDF is created from a scanned document, then the text in the PDF is essentially a picture and not text that can be copied and pasted. In my case I receive these PDF scans from missionaries’ prayer letters that need to be turned into blog posts or used in newsletters. I want to copy the text without having to retype the whole letter.

Setup Information

I am using Linux as the OS. The main software I am using to do the heavy lifting is Tesseract OCR. They have a Windows version. You can probably figure out a way to make most of these tools (or equivalents) work in a Windows environment. But, if you are using Windows, you probably don’t do this geeky kind of stuff. You are still probably retyping any document you need to do something like this on..

Besides Tesseract OCR, I am using ImageMagick to do image conversion. They also have a Windows version of their program.

Steps

You need to take the original PDF and convert it into an image file using ImageMagick. But, it is not as simple as issuing the convert command. You have to give it a couple of other parameters. One is that the file must be an 8 bit color scheme or Tesseract will choke on it. Also it needs to be scaled up to sufficient dpi (dots per inch). ImageMagick’s convert command will output a 72 dpi file by default. My scanner scans at 300 dpi by default, so I can easily convert the PDF to a 300 dpi image which is enough to get a decent OCR output.

Details

CD into the directory where your PDF is or you will need to add the paths to the following commands.

Convert PDF

The string equals: use imagemagick to create a 300 dpi image at a color depth of 8 bits from file.pdf into a file named file.tiff in the current folder.

Run Tesseract OCR on file.tiff

This string equals: Do OCR (optical character recognition) using Tesseract on file.tiff and output it to a file called OutputFileName.txt in the same folder.

Future Project

I plan to turn this into a Python script to simplify this into a single step [it became a bash script instead]. I am learning Python at the moment and don’t know all the pieces I need to know to make the script. But, ultimately I will use Python to do RegEx (regular expression) find and replace on the end of lines so that paragraphs are maintained in the final output and there are not a large number of forced line breaks. Then I can just open the .txt file in a text editor and copy and paste the contents into they website.

[The blog post from Kiirani that put me on the right track.]