Jan 15, 2021
Great write up! It probably needs a more motivating example, as the AirBnb file can be loaded using --allow_quoted_newlines option.
bq load --source_format CSV --autodetect --allow_quoted_newlines tmp.airbnb ~/Downloads/AB_NYC_2019.csv
BigQuery supports quoted new lines, one just has to tell BigQuery the file uses them. By default BigQuery assumes the file does not have new lines, and this allows parallel load even of a single large CSV file.