3 ms·

Suffice to say that the 600MB just lets you draw the roads on a plane, it's like comparing an ascii art drawing of the road (from .csv/shp) to a digital still o

Suffice to say that the 600MB just lets you draw the roads on a plane, it's like comparing an ascii art drawing of the road (from .csv/shp) to a digital still of the road (the amount of information you'd actually need). you absolutely cannot rely on "a couple of sensor [types]". I mentioned i have nearly a half million miles on the road. All of that prior experience influences my driving when i am driving someplace new. in that 8TB disk, you have to find a way to produce that "experience", except instead of my 0.5mm miles, you are talking about the aggregate "experience" of 0.5mm miles per road per unit of time (day for some places like I-10 through Los Angeles, month for others, maybe a year for some "rural" roads.) none of this has to do with visual or proprioception. It's knowing "every inch" of road. It's knowing how far i can leave the center of the lane if someone else crowds me or goes over the center divider, because the shoulder is soft through here because logging trucks have been exiting the forest onto the highway. It's knowing what part of I-605 floods - not the whole thing, some lanes, some places, and "flood" means 2+ inches of water on the road surface, hitting it at speed makes a tidal wave flying into other lanes. If someone hits that in front of you, you're blind for a couple of seconds minimum. If we want to have semi trucks be "FSD" it needs to know, for the traffic and other conditions, how fast to go and what gear to be in to climb each hill, and then the hazards that are over the hill - that a trucker would know. Where's the gravel bed on more mountainous passes? Or more simply, what time of day neighborhoods are more likely to have people approaching or going through / out of intersections, blind or otherwise. How many "bytes" is that information, times every neighborhood? If many cars brake at the same place, there's probably a reason, and that needs to be either in the dataset or updated somehow if conditions change. You ever used Waze and had a report of something on the road or a cop parked somewhere, and it's nowhere to be seen? And that's updated much more frequently than the radio-info on the GPS systems i referred to earlier. Some roads become impassable in the rain, some roads ice more readily. If this was easy/simple/solved, waymo et al would be bragging about it, the tech in their cars. Waymo (or the other one) specifically, because they cover less than 0.1% of road surfaces in the US, in some of the most maintained and heavily traveled corridors in the world. So, if anyone from a robotaxi company happens by and knows roughly how much storage is needed for <0.1% of the road surfaces in the US, then we could actually start to have this dialog in a meaningful way. Also i am unsure how much coverage robotaxis actually have in their service area. A "grid system" of roads makes mapping and aggregate data "simple", for sure. This reminded me a bit of the idea that somewhere in the US there's a database of every sms sent to or from US cellular phones. "it's just text; it'll compress well" - belies how much text there is, there. for reference, the map in my lexus is ~8GB, for the US. And that's just "shapes" and POI and knowing how the addressing works on each road. It doesn't know what lane i'm in, it doesn't track curves in the road effectively (the icon leaves the road while i'm driving quite often), and overpasses and the like confuse all GPS systems i've ever used - like in Dallas, TX where it's 4 layers high and parallel roads stacked. furthermore, just the road data on google maps for the nearest metro area to my house is 20MB. i have a recollection it goes real quick into hundreds of MB if you need to download maps for the swaths of areas where there is no cellphone reception, like areas in western Nevada. given 20MB for my metro, that's 40GB of just road shapes and addresses for the US, which is much more than the 600MB incomplete GIS files i downloaded. so we've moved from fencing 600MB "text" data; to the actual data needed by a GPS to give directions, 8000MB. Your claim is that a mere 1000x more data is enough to autonomously self-drive anywhere in the US, at any time of day or year, etc... you know who actually has this data and would know how big it is? Tesla.

Dylan16807 1y ago
> prior experience The part of the computer that knows how to drive is completely separate from the 7TB database of the exact shape and location of every lane and edge and defect. > knowing how far i can leave the center of the lane if someone else crowds me or goes over the center divider Experience, not in the database. > knowing what part of I-605 floods > Where's the gravel bed on more mountainous passes? That goes in the database but it's less than one byte per meter. > How many "bytes" is that information, times every neighborhood? I don't know why you would want that data, you should be wary of blind traffic at all times, but that's easy math. There's less than a million neighborhoods and time based activity levels for each neighborhood would be about a hundred bytes. So: Less than 1 byte per meter and less than 100MB total. > If this was easy/simple/solved, waymo et al would be bragging about it This doesn't happen for two reasons. One they are collecting orders of magnitude more data than road info, two like I keep saying the collection is extremely difficult and I'm only defending the storage and use as being feasible. > This reminded me a bit of the idea that somewhere in the US there's a database of every sms sent to or from US cellular phones. "it's just text; it'll compress well" - belies how much text there is, there. Well we know how many meters of road there are. So it's basic multiplication. I can tell you how many hard drives you need to store a trillion texts. It's five hard drives. Google thinks the human race sends almost ten trillion text messages per year. So I guess you could store them all very easily? Why do you think it's not doable? > Your claim is that a mere 1000x more data is enough to autonomously self-drive anywhere in the US, at any time of day or year, etc... My claim is that 1000x is enough for utterly exhaustive road maps. Figuring out how to drive is another thing entirely.
- genewitch 1y ago
  ohhhh, we're arguing past eachother. I am unsure how to reconcile. an SMS isn't just "140 characters/bytes" or whatever (i honestly don't care what your definition of "SMS" is). Of course you could fit 140 characters * 1e12 onto 5 hard drives. Where are you going to put the 1PB (for 1e12, but your own cite says it's 1e13, so 10PB) of metadata, minimum? the most barebones amount of metadata you need to actually have actionable "intelligence" is 1KB per message (technically i was able to finagle it to ~1016 bytes.) And that's for every message, even an SMS that is the single character "K". you need the metadata to derive any information from the SMS. "Lunch?" "yeah" "where?" "the place with the wheel" "okay see you in 25, bring Joel" This is what you propose to save. (quick math shows you went off something like ~32TB of sms data per 1e12 messages) in the same way that you propose that the shapes of a road and it's direction and distance "plus 1KB of metadata per meter" is enough to derive the ability to drive upon those roads. It's pretty obvious that just using sensors is not going to get FSD. Maybe in the next 20 years we will develop sensor technology (and swarm networking and whatever else) that will allow us to dispense with the "7TB" of metadata. My argument is that: we need much more "metadata" than 1KB per meter to "know the road baseline, current conditions, hazards", much in the same way a text message is more than 140 bytes. Driving with "only sensors" and rough GPS has killed people. It does not matter if human drivers have more death per million miles or whatever, because i am strictly talking about FSD, what other people are calling level 5 (i'd even concede level 4; although i wouldn't be able to use a level 4 car where i live for roughly 1/4th the year - and other areas would have more than 1/4th the year.) enjoy your night! note: the metadata for a meter of road could be: { "road_segment": { "segment_id": 3500000, "meter_position": 128534, "coordinates": { "latitude": 32.5385, "longitude": -92.9222 }, "timestamp_added": "2024-06-05T23:57:34Z", "last_updated": { "timestamp": "2025-06-05T23:57:36Z", "delta_seconds": 2.0 }, "hash_signature": "89a25b6f3cd829e671bb9d42e8fae2c6", "road_type": "highway", "lane_data": { "lane_count": 4, "lane_width_m": 3.5, "shoulder_width_m": 2.0, "divider_type": "concrete barrier", "markings": ["solid white", "dashed white", "double yellow"] }, "speed_limit_mph": 65, "road_material": "asphalt", "incline_percent": 1.2, "curve_radius_m": 150, "surface_condition": "dry", "weather_conditions": { "timestamp": "2025-06-05T23:57:36Z", "temperature_c": 25, "precipitation_mm": 0, "visibility_m": 5000, "wind_speed_mps": 2.5 }, "baseline_hazards": [ { "type": "grade_crossing", "description": "Railroad crossing with signal lights", "location": { "latitude": 32.5386, "longitude": -92.9224 } }, { "type": "roadwork", "description": "Permanent lane narrowing from past construction", "location": { "latitude": 32.5390, "longitude": -92.9230 } } ], "current_hazards": [ { "type": "construction", "description": "Active roadwork zone with lane closure", "severity": "high", "location": { "latitude": 32.5387, "longitude": -92.9226 } }, { "type": "downed_power_line", "description": "Reported electrical hazard near shoulder", "severity": "critical", "location": { "latitude": 32.5395, "longitude": -92.9232 } } ] } } Obviously you can reduce this, but there's a minimum viable amount of metadata, that's my claim, and it's more than 1KB per meter. that snippet is ~1800bytes as is. the "current conditions" would not be part of the dataset on the "7TB" disk. that would need to be relayed or otherwise ingested by the car as it drives - the way my 2012 lexus tells me that i'm about to drive into a wild storm, but that's all the extra information i get out of its infotainment system. waze is a better example of the sort of realtime updates i expect a FSD to need; although i expect many times more points of information than waze has, maybe dozens, maybe hundreds more. and each "trick" you do to reduce the size of the metadata necessarily implies more CPU needed to parse and process it.

Dylan16807 1y ago

> prior experience The part of the computer that knows how to drive is completely separate from the 7TB database of the exact shape and location of every lane and edge and defect. > knowing how far i can leave the center of the lane if someone else crowds me or goes over the center divider Experience, not in the database. > knowing what part of I-605 floods > Where's the gravel bed on more mountainous passes? That goes in the database but it's less than one byte per meter. > How many "bytes" is that information, times every neighborhood? I don't know why you would want that data, you should be wary of blind traffic at all times, but that's easy math. There's less than a million neighborhoods and time based activity levels for each neighborhood would be about a hundred bytes. So: Less than 1 byte per meter and less than 100MB total. > If this was easy/simple/solved, waymo et al would be bragging about it This doesn't happen for two reasons. One they are collecting orders of magnitude more data than road info, two like I keep saying the collection is extremely difficult and I'm only defending the storage and use as being feasible. > This reminded me a bit of the idea that somewhere in the US there's a database of every sms sent to or from US cellular phones. "it's just text; it'll compress well" - belies how much text there is, there. Well we know how many meters of road there are. So it's basic multiplication. I can tell you how many hard drives you need to store a trillion texts. It's five hard drives. Google thinks the human race sends almost ten trillion text messages per year. So I guess you could store them all very easily? Why do you think it's not doable? > Your claim is that a mere 1000x more data is enough to autonomously self-drive anywhere in the US, at any time of day or year, etc... My claim is that 1000x is enough for utterly exhaustive road maps. Figuring out how to drive is another thing entirely.

genewitch 1y ago
ohhhh, we're arguing past eachother. I am unsure how to reconcile. an SMS isn't just "140 characters/bytes" or whatever (i honestly don't care what your definition of "SMS" is). Of course you could fit 140 characters * 1e12 onto 5 hard drives. Where are you going to put the 1PB (for 1e12, but your own cite says it's 1e13, so 10PB) of metadata, minimum? the most barebones amount of metadata you need to actually have actionable "intelligence" is 1KB per message (technically i was able to finagle it to ~1016 bytes.) And that's for every message, even an SMS that is the single character "K". you need the metadata to derive any information from the SMS. "Lunch?" "yeah" "where?" "the place with the wheel" "okay see you in 25, bring Joel" This is what you propose to save. (quick math shows you went off something like ~32TB of sms data per 1e12 messages) in the same way that you propose that the shapes of a road and it's direction and distance "plus 1KB of metadata per meter" is enough to derive the ability to drive upon those roads. It's pretty obvious that just using sensors is not going to get FSD. Maybe in the next 20 years we will develop sensor technology (and swarm networking and whatever else) that will allow us to dispense with the "7TB" of metadata. My argument is that: we need much more "metadata" than 1KB per meter to "know the road baseline, current conditions, hazards", much in the same way a text message is more than 140 bytes. Driving with "only sensors" and rough GPS has killed people. It does not matter if human drivers have more death per million miles or whatever, because i am strictly talking about FSD, what other people are calling level 5 (i'd even concede level 4; although i wouldn't be able to use a level 4 car where i live for roughly 1/4th the year - and other areas would have more than 1/4th the year.) enjoy your night! note: the metadata for a meter of road could be: { "road_segment": { "segment_id": 3500000, "meter_position": 128534, "coordinates": { "latitude": 32.5385, "longitude": -92.9222 }, "timestamp_added": "2024-06-05T23:57:34Z", "last_updated": { "timestamp": "2025-06-05T23:57:36Z", "delta_seconds": 2.0 }, "hash_signature": "89a25b6f3cd829e671bb9d42e8fae2c6", "road_type": "highway", "lane_data": { "lane_count": 4, "lane_width_m": 3.5, "shoulder_width_m": 2.0, "divider_type": "concrete barrier", "markings": ["solid white", "dashed white", "double yellow"] }, "speed_limit_mph": 65, "road_material": "asphalt", "incline_percent": 1.2, "curve_radius_m": 150, "surface_condition": "dry", "weather_conditions": { "timestamp": "2025-06-05T23:57:36Z", "temperature_c": 25, "precipitation_mm": 0, "visibility_m": 5000, "wind_speed_mps": 2.5 }, "baseline_hazards": [ { "type": "grade_crossing", "description": "Railroad crossing with signal lights", "location": { "latitude": 32.5386, "longitude": -92.9224 } }, { "type": "roadwork", "description": "Permanent lane narrowing from past construction", "location": { "latitude": 32.5390, "longitude": -92.9230 } } ], "current_hazards": [ { "type": "construction", "description": "Active roadwork zone with lane closure", "severity": "high", "location": { "latitude": 32.5387, "longitude": -92.9226 } }, { "type": "downed_power_line", "description": "Reported electrical hazard near shoulder", "severity": "critical", "location": { "latitude": 32.5395, "longitude": -92.9232 } } ] } } Obviously you can reduce this, but there's a minimum viable amount of metadata, that's my claim, and it's more than 1KB per meter. that snippet is ~1800bytes as is. the "current conditions" would not be part of the dataset on the "7TB" disk. that would need to be relayed or otherwise ingested by the car as it drives - the way my 2012 lexus tells me that i'm about to drive into a wild storm, but that's all the extra information i get out of its infotainment system. waze is a better example of the sort of realtime updates i expect a FSD to need; although i expect many times more points of information than waze has, maybe dozens, maybe hundreds more. and each "trick" you do to reduce the size of the metadata necessarily implies more CPU needed to parse and process it.