Holzer was born in Vienna, Austria. His interest in the supernatural was sparked, at a young age, by stories told to him by his uncle Henry. He went on to study archaeology and ancient history at the University of Vienna but sensing that war was imminent, his family decided it was unsafe to stay in Austria and left the country for New York City in 1938. He studied japanese at Columbia University and comparative religion and parapsychology at London College of Applied Science - obtaining a Ph.D. He went on to teach parapsychology at the New York Institute of Technology.