The term CGI (Common Gateway Interface) means that the generated page isn't a script controlled by the HTTP daemon such as apache but is generated externally by the scripting language and then given back to the HTTP daemon to give to the user. For the "non CGI" there is a module loaded into the daemon and the daemon controls the process, this is usually true for PHP but doesn't need to be. This Common Gateway Interface - Wikipedia, the free encyclopedia is a good resource for you to explore.

It was quite common in the 90s to have dynamic elements of a site in CGI binaries (files) located in IANA — Example domains - sometimes they'd be scripts interpretted on-the-fly (like Perl), but often they'd be pre-compiled binaries (like Windows executables) because these usually run much faster due to being compiled.