Add -o Acquire::ForceIPv4=true when running apt-get.

If you want to make the setting persistent just create /etc/apt/apt.conf.d/99force-ipv4 and put Acquire::ForceIPv4 "true"; in it:

echo 'Acquire::ForceIPv4 "true";' | sudo tee /etc/apt/apt.conf.d/99force-ipv4

IPv6 can also cause a problem when running the Linux installer. To fix this get the IPv4 address of the repository and specify it manually:

ping -4

Then select "enter information manually" for the mirror. Just enter the IP address.