The person directly responsible for the nominal pipe size was a gentleman by the name of Robert Briggs. Briggs was the superintendent of the Pascal Iron Works in Philadelphia. In 1862, he wrote a set of pipe specifications for iron pipe, and passed them around to all of the mills in the area.
Realize that in 1862, this country was engaged in the Civil War. Each pipe mill made its own pipe and fittings to its own specifications. Briggs tried to standardize the sizing, which would also help the war effort. The pipe and fittings would be interchangeable between mills. This was rather novel in 1862.