CartoDBPerformance - UCL-ShippingGroup/shipviz GitHub Wiki

Some test files with a different numbers of rows can be found here.

Note: Files > 100MB can't be uploaded to GitHub, so the biggest csv files can't be uploaded.

Note: Configuration on max rows, max file size, max concurrent ingestiongs, etc can be changed with:

 rake cartodb:set_custom_limits_for_user[development,157286400000,10000000,10] 

No need to restart server or anything.

I'm testing several ingestions and these are the results so far:

Points	Vessels	Ingestion time	Vis Quality	CartoDB dataset size	Comments	File
100K	?	1m	Very Good	?		vessels100K.csv
200K	?	1.5m	Very Good	?		vessels200K.csv
300K	?	2m	Very Good	?		vessels300K.csv
400K	?	2.5m	Very Good	62MB		vessels400K.csv
500K	?	3m	Good	78MB		vessels500K.csv
600K	?	3.5m	Good	?	Gave "Too many rows error", had to change settings with rake	vessels600K.csv
700K	188	4m	Slowish to load	?		vessels700K.csv
800K	202	4.5m	Slowish to load	109MB		vessels800K.csv
900K	218	5m	Slowish to load	126MB		vessels900K.csv
1000K	237	5.5m	Slowish to load	154MB		vessels1000K.csv
1500K	328	7m	Slow to load	226MB	Some tiles empty unless zooming in	vessels1500K.csv
2000K	400	9m	Slow to load	300MB	Some tiles empty unless zooming in	vessels2000K.csv
2500K	531		Slow to load	378MB	Some tiles don't show all the data	vessels2500K.csv
3000K	636		Slow to load	453MB	Some tiles don't show all the data	vessels3000K.csv
4000K	936	20m	Slow to load	635MB	missing data unless zooming in - unreliable	vessels4000K.csv
4500K						vessels4500K.csv
5000K		25m	Slow to load		missing data unless zooming in - unreliable	vessels5000K.csv