4 ms·

Josh from the Community Broadband Project here. Would be happy to answer any questions about our mission and progress in LA. I'm no regular, but we just got a b

Josh from the Community Broadband Project here. Would be happy to answer any questions about our mission and progress in LA. I'm no regular, but we just got a bunch of traffic from Hacker News and I'm here to spread the gospel. NYC Mesh is doing very cool stuff--we wish them the best. We're in touch with a few other community ISPs who are growing organically in competition against big incumbent ISPs. It's inspiring. Beverly Hills, West Hollywood, and Culver City are all constructing municipal fiber right now. In the case of at least WeHo and Culver, it's aimed primarily at businesses. Some residents may see fiber offered by the incumbents, but we did some math of those offerings and it was $133/mo for 1Gbps. It'd be more helpful for the community if reasonable speeds were more affordable and available, than only some wealthy neighborhoods getting fiber.

teleclimber 8y ago
Hi, is your product based on the Necto offering? https://apply.nectolab.io https://apply.nectolab.io
- fixthenet 8y ago
  No, but Necto looks like a very interesting way to make ISPs more approachable for a community. That's one of our own goals, but in our case we're doing everything in house, which makes the most sense given our expertise.
debaserab2 8y ago
You might be the totally wrong person to ask this and I apologize if this is a naive question. Is there a way a lay person (non network-engineer) can somehow get involved with their community to help get this started? I'd love to see something like this get started where I live (Minneapolis) and would be willing to help in any capacity I could.
- fixthenet 8y ago
  Like any business endeavor, if you don't have the skills to do it yourself, you need the resources to gain the skills, either through learning, networking and meeting people, or hiring someone. The scale you start at changes things too. We've spoken to a few grassroots ISPs that started 3-5 years ago by non-IT folks who just wanted to share with a few neighbors. Over the course of time they learned and grew the network and now they're at a couple hundred customers. Not big in the ISP sense, but it provided an essential service to their community and competes with [and steals customers from] incompetent, or at worse, anti-consumer incumbent ISPs. If you have the passion, but not the IT skill set, be the banner waver. Be the leader of the endeavor. Use your passion to convert others. You'll find your talent.
- toomuchtodo 8y ago
  Check out https://www.muninetworks.org https://www.muninetworks.org.
- walrus01 8y ago
  One way that a lay person can help is with rooftop site acquisition, negotiation, contracts, marketing and presentation materials. Or as labor to install stuff, if it is a real nonprofit. Own a van that you can tie a 18' extension ladder to the roof? Great, you get to be the guy buying 16" concrete blocks from home Depot for the roof mount ballast.
instaheat 8y ago
Hi Josh. Forgive my ignorance, I am honestly just curious. I took a look at your test area on your website, and it is filled with homes/condos that are in the millions of dollars. What practical reason would someone that could afford a home that costs this much need to utilize your service? Is this area essentially exactly that, a test area, and not intended to be the location you deploy in first?
- fixthenet 8y ago
  Our intended initial deployment was in Culver City, off a tower that had reach to Culver and city district 10. We met with LA city, Culver City, and did a lot of prep in that direction. Unfortunately, the critical path to fiber was delayed in construction for a year, with another year to go. In addition, we were thrown a curve ball with a tower engineering study prior to that tower's use that would eat up a sizable portion of funding. Right around this time we were offered funding from WeTransfer as part of their Net Neutrality campaign, but under the obligation serve their area in Venice. We decided to go for it, and use that as a launching point for a proof of concept network. While not ideal for obvious reasons, it goes a long way towards proving our tech and business model. Our hope is that we can grow out the network towards Culver and CD10, acquire some critical infrastructure there, and then push east across the West Adams/Jefferson Park corridor. It is a true deployment--we don't plan to uproot the Venice service, but it may end up being an arm of the network as opposed to the core in the future. One of the key goals of our project is to free low income neighborhoods from stone age technology by offering broadband speeds far above DSL for a fair price.
  - fixthenet 8y ago
    I should add that it'll take us some time to tap into the federal programs and grants to truly make this incredible for low income neighborhoods. Venice has been serving us well as a means to get our sea legs, so to speak. Starting there won't impact our plans for closing the technology gap where it exists.
  - instaheat 8y ago
    Thanks for the thoughtful reply. I appreciate it - makes sense! I wish you all the best of luck.
ucaetano 8y ago
Hi Josh! Do you buy any transit at all or rely solely on peering? Who do you peer with, and how easy has it been to find peers, due to the huge asymmetry in traffic? Do you have any CDN caches deployed (GGC, Netflix, etc.)?
- toast0 8y ago
  I'm not related, but any small ISP is going to need transit. Larger ISPs have pretty high requirements to peer, but random sites will be single homed in those networks; to say nothing of overseas sites. But, most of the networks that provide the majority of the traffic have pretty open peering policies, and the LA area has a lot of places to peer, and peering places are also usually a reasonable place to get transit so... If you're seeing at least a few gbps from FAANG, they'll peer with you, if you're where they are, and all of them have PoPs in LA, because everyone is there (this is a little circular, LA was highly connected in the PTSN because of population size, and that lead to early internet exchanges and undersea cabling, which leads to more people wanting to be there and so on) I'd imagine if they have the traffic to justify it, and the space to house it, they'd get the CDN caches. It's a pretty good win, peering is settlement free, but access to the peering point isn't free; keeping traffic from using your ingress capacity is nice (depending on your network architecture anyway)
- fixthenet 8y ago
  Great tech questions. Right now, in our beta/test network, we're receiving DIA service to a remote server location and that's it. No peering, no CDNs. Our next steps are to increase the bandwidth of our DIA, since that's more cost effective at our current scale than we'd benefit by peering. The step after that is to swap that DIA for transport and then colocate in DTLA to find peers. CDN caches often require a minimum number of customers on network, something like 5000 for Netflix, if I recall, so we'll have to grow before we can take advantage of caching.
  - amazon_not 8y ago
    A Netflix cache appliance isn't dependent on the number of customers, it's on dependent on the amount of traffic from Netflix. You don't need 5000 Netflix customers, you need 5 Gbps of peak traffic to qualify. However that isn't your only option. You can just peer with Netflix, they have an open peering policy.
Timmah 8y ago
I don't know if laying cable is one of your biggest challenges, but I assume it would be along with right-of-way. What do you think about the idea of using the sewer system to lay either fiber or 10+gb copper? Before you think about how gross that is, consider the advancements in drone robotics. You would need to devise a cable-laying robot that could traverse the sewer via remote control. Mount fixtures to the top of the pipes and string the cables through, connecting neighborhoods. This would require convincing the city to lease you the right-of-way and put up a bond to cover any repairs to damage caused by the cables. But it would solve both the bandwidth problem and the "those are our telephone poles!" Problem. What do you think?
- amazon_not 8y ago
  Rule #1: Never install new copper in the outside plan. That being said, there are companies specializing in installing fiber cables into sewers. They use tracked robots to install steel bands inside the pipes that keep the fiber cable in place. It's not cheap, but it is cheaper than ripping open the streets in a city. There are also techniques to install fiber cable into water pipes.
- SECProto 8y ago
  Water utilities are (justifiably) very protective of their infrastructure. Justifiable because of the extreme expense in closing a street, excavating, and repairing anything that happens to their buried infrastructure. I've seen the one in my city reject a stormwater pipe because of a couple small pieces of crusher dust. I can't see them being positive towards the deliberate installation of another biofilm-forming-surface inside their pipes. But YMMV I'm only speaking from experience in my small neck of the woods!
  - Timmah 8y ago
    I don't disagree. I think that a full mitigation plan could help. For example showing that fiber can be easily chopped up if a pipe clogged. Maybe biofilm could be mitigated with an electric charge on a conductive outer sheath? I can't see how a single cable or 2 would ever require digging into the street.

teleclimber 8y ago

Hi, is your product based on the Necto offering? https://apply.nectolab.io https://apply.nectolab.io

fixthenet 8y ago
No, but Necto looks like a very interesting way to make ISPs more approachable for a community. That's one of our own goals, but in our case we're doing everything in house, which makes the most sense given our expertise.

debaserab2 8y ago

You might be the totally wrong person to ask this and I apologize if this is a naive question. Is there a way a lay person (non network-engineer) can somehow get involved with their community to help get this started? I'd love to see something like this get started where I live (Minneapolis) and would be willing to help in any capacity I could.

fixthenet 8y ago
Like any business endeavor, if you don't have the skills to do it yourself, you need the resources to gain the skills, either through learning, networking and meeting people, or hiring someone. The scale you start at changes things too. We've spoken to a few grassroots ISPs that started 3-5 years ago by non-IT folks who just wanted to share with a few neighbors. Over the course of time they learned and grew the network and now they're at a couple hundred customers. Not big in the ISP sense, but it provided an essential service to their community and competes with [and steals customers from] incompetent, or at worse, anti-consumer incumbent ISPs. If you have the passion, but not the IT skill set, be the banner waver. Be the leader of the endeavor. Use your passion to convert others. You'll find your talent.
toomuchtodo 8y ago
Check out https://www.muninetworks.org https://www.muninetworks.org.
walrus01 8y ago
One way that a lay person can help is with rooftop site acquisition, negotiation, contracts, marketing and presentation materials. Or as labor to install stuff, if it is a real nonprofit. Own a van that you can tie a 18' extension ladder to the roof? Great, you get to be the guy buying 16" concrete blocks from home Depot for the roof mount ballast.

instaheat 8y ago

Hi Josh. Forgive my ignorance, I am honestly just curious. I took a look at your test area on your website, and it is filled with homes/condos that are in the millions of dollars. What practical reason would someone that could afford a home that costs this much need to utilize your service? Is this area essentially exactly that, a test area, and not intended to be the location you deploy in first?

fixthenet 8y ago
Our intended initial deployment was in Culver City, off a tower that had reach to Culver and city district 10. We met with LA city, Culver City, and did a lot of prep in that direction. Unfortunately, the critical path to fiber was delayed in construction for a year, with another year to go. In addition, we were thrown a curve ball with a tower engineering study prior to that tower's use that would eat up a sizable portion of funding. Right around this time we were offered funding from WeTransfer as part of their Net Neutrality campaign, but under the obligation serve their area in Venice. We decided to go for it, and use that as a launching point for a proof of concept network. While not ideal for obvious reasons, it goes a long way towards proving our tech and business model. Our hope is that we can grow out the network towards Culver and CD10, acquire some critical infrastructure there, and then push east across the West Adams/Jefferson Park corridor. It is a true deployment--we don't plan to uproot the Venice service, but it may end up being an arm of the network as opposed to the core in the future. One of the key goals of our project is to free low income neighborhoods from stone age technology by offering broadband speeds far above DSL for a fair price.
- fixthenet 8y ago
  I should add that it'll take us some time to tap into the federal programs and grants to truly make this incredible for low income neighborhoods. Venice has been serving us well as a means to get our sea legs, so to speak. Starting there won't impact our plans for closing the technology gap where it exists.
- instaheat 8y ago
  Thanks for the thoughtful reply. I appreciate it - makes sense! I wish you all the best of luck.

ucaetano 8y ago

Hi Josh! Do you buy any transit at all or rely solely on peering? Who do you peer with, and how easy has it been to find peers, due to the huge asymmetry in traffic? Do you have any CDN caches deployed (GGC, Netflix, etc.)?

toast0 8y ago
I'm not related, but any small ISP is going to need transit. Larger ISPs have pretty high requirements to peer, but random sites will be single homed in those networks; to say nothing of overseas sites. But, most of the networks that provide the majority of the traffic have pretty open peering policies, and the LA area has a lot of places to peer, and peering places are also usually a reasonable place to get transit so... If you're seeing at least a few gbps from FAANG, they'll peer with you, if you're where they are, and all of them have PoPs in LA, because everyone is there (this is a little circular, LA was highly connected in the PTSN because of population size, and that lead to early internet exchanges and undersea cabling, which leads to more people wanting to be there and so on) I'd imagine if they have the traffic to justify it, and the space to house it, they'd get the CDN caches. It's a pretty good win, peering is settlement free, but access to the peering point isn't free; keeping traffic from using your ingress capacity is nice (depending on your network architecture anyway)
fixthenet 8y ago
Great tech questions. Right now, in our beta/test network, we're receiving DIA service to a remote server location and that's it. No peering, no CDNs. Our next steps are to increase the bandwidth of our DIA, since that's more cost effective at our current scale than we'd benefit by peering. The step after that is to swap that DIA for transport and then colocate in DTLA to find peers. CDN caches often require a minimum number of customers on network, something like 5000 for Netflix, if I recall, so we'll have to grow before we can take advantage of caching.
- amazon_not 8y ago
  A Netflix cache appliance isn't dependent on the number of customers, it's on dependent on the amount of traffic from Netflix. You don't need 5000 Netflix customers, you need 5 Gbps of peak traffic to qualify. However that isn't your only option. You can just peer with Netflix, they have an open peering policy.

Timmah 8y ago

I don't know if laying cable is one of your biggest challenges, but I assume it would be along with right-of-way. What do you think about the idea of using the sewer system to lay either fiber or 10+gb copper? Before you think about how gross that is, consider the advancements in drone robotics. You would need to devise a cable-laying robot that could traverse the sewer via remote control. Mount fixtures to the top of the pipes and string the cables through, connecting neighborhoods. This would require convincing the city to lease you the right-of-way and put up a bond to cover any repairs to damage caused by the cables. But it would solve both the bandwidth problem and the "those are our telephone poles!" Problem. What do you think?

amazon_not 8y ago
Rule #1: Never install new copper in the outside plan. That being said, there are companies specializing in installing fiber cables into sewers. They use tracked robots to install steel bands inside the pipes that keep the fiber cable in place. It's not cheap, but it is cheaper than ripping open the streets in a city. There are also techniques to install fiber cable into water pipes.
SECProto 8y ago
Water utilities are (justifiably) very protective of their infrastructure. Justifiable because of the extreme expense in closing a street, excavating, and repairing anything that happens to their buried infrastructure. I've seen the one in my city reject a stormwater pipe because of a couple small pieces of crusher dust. I can't see them being positive towards the deliberate installation of another biofilm-forming-surface inside their pipes. But YMMV I'm only speaking from experience in my small neck of the woods!
- Timmah 8y ago
  I don't disagree. I think that a full mitigation plan could help. For example showing that fiber can be easily chopped up if a pipe clogged. Maybe biofilm could be mitigated with an electric charge on a conductive outer sheath? I can't see how a single cable or 2 would ever require digging into the street.