Would community-owned data centers help with the sovereignty of data?

1

Minio

The capability already exists to run data infrastructure without public clouds, as this is what everyone did before public clouds. You either rented space in a data center to put your own hardware or you ran it from your physical offices (many Wall St firms still do). Currently, the Lakehouse architecture is gaining a lot of popularity in the data engineering world and it lends itself quite nicely to self custody of data. The core thought of this architecture is that the data lives separately than the CPUs that are going to perform computations. A query of engine of some kind, like DuckDB, Clickhouse, or Spark, runs on your local hardware and downloads the data from the remote location whenever you need it. Most of these tools can talk to any storage system that is compatible with the S3 XML API, which is Amazon's storage API. With a tool like Minio, its possible to turn any server into an S3 compatible storage system. A company can store all its data in a few servers at their office, and allow employees to use their laptops (using say DuckDB) to do all the computations. MotherDuck is a startup in this area that sounds like it might be doing some work to productize this kind of thing.

#Cloud Storage #Cloud Computing #Object Storage 154 social mentions

2

FileCoin

The reality is that running the hardware yourself tends to be a bad business decision. Businesses are incentivized to only spend time/energy/money on things that are going to differentiate their product. Running your own storage layer is possible, but often more expensive or time consuming than just paying Amazon or Google. There has been a push in the world of Web3 (if you believe in that kind of thing) to build decentralized competitors. Filecoin is probably the most well known. The general idea is that instead of paying Google or Amazon to run these servers, you can pay individuals who have extra space on their servers. All of that is to be managed and paid for via blockchains.

#Blockchain #Cloud Storage #Decentralized Storage 75 social mentions

3

Amazon S3

The capability already exists to run data infrastructure without public clouds, as this is what everyone did before public clouds. You either rented space in a data center to put your own hardware or you ran it from your physical offices (many Wall St firms still do). Currently, the Lakehouse architecture is gaining a lot of popularity in the data engineering world and it lends itself quite nicely to self custody of data. The core thought of this architecture is that the data lives separately than the CPUs that are going to perform computations. A query of engine of some kind, like DuckDB, Clickhouse, or Spark, runs on your local hardware and downloads the data from the remote location whenever you need it. Most of these tools can talk to any storage system that is compatible with the S3 XML API, which is Amazon's storage API. With a tool like Minio, its possible to turn any server into an S3 compatible storage system. A company can store all its data in a few servers at their office, and allow employees to use their laptops (using say DuckDB) to do all the computations. MotherDuck is a startup in this area that sounds like it might be doing some work to productize this kind of thing.

#Cloud Hosting #Object Storage #Cloud Storage 171 social mentions