22 ms·

Google Cloud Storage FUSE

149 points by mvolfik 3y ago

rippercushions 3y ago
Is this the same gcsfuse that's been around for years, only now with official Google support? https://github.com/GoogleCloudPlatform/gcsfuse https://github.com/GoogleCloudPlatform/gcsfuse
- beastman82 3y ago
  yes > export GCSFUSE_REPO=gcsfuse-`lsb_release -c -s`
- scoobydoobydrew 3y ago
  Looking at change descriptions, looks like underlying changes were made to get to this like now using GO client library. I would expect a more stable product, and better performance which looks like the performance benchmarks located under docs has been updated as well. Happy to finally see Google standing behind this, and the official CSI driver is really cool to see.
- throwdbaaway 3y ago
  Heh, my old laptop has a git clone of this from September 1st 2016.
trollied 3y ago
Be aware that this is not free: "Cloud Storage FUSE is available free of charge, but the storage, metadata, and network I/O it generates to and from Cloud Storage are charged like any other Cloud Storage interface. In other words, all data transfer and operations performed by Cloud Storage FUSE map to Cloud Storage transfers and operations, and are charged accordingly."
- rippercushions 3y ago
  Using FUSE doesn't cost you anything extra, but it doesn't make the underlying storage free.
  - ElectricalUnion 3y ago
    You will be doing storage operations silently and in a unoptimized fashion, more so if the underlying FUSE filesystem is implemented in a naive fashion. For example, Cloud Storage never moves or renames your objects; copying and deleting the original one instead. This can end up costing quite a lot if you're using data other that in "standard store" because of minimum storage duration.
jefftk 3y ago
Cloud Storage FUSE does not support overwriting in the middle of a file. Only sequential writes are supported. This seems like a big limitation?
- throwawaaarrgh 3y ago
  pretty standard limitation of object storage services iirc
  - jefftk 3y ago
    Doesn't this mean that most programs you might want to use with the FUSE API won't actually work? They'll do fine for a while, until they try to seek, and then they'll get an error? Or is there a large group of programs that only ever write sequentially?
    - jsnell 3y ago
      I'd think non-appending writes are quite rare in practice, other than databases. Even when the application is logically overwriting data, in other kinds of programs it's almost always implemented as writing to a new file + an atomic rename, not in-place modification.
      - dontlaugh 3y ago
        Even databases tend to do sequential writes, whether to a WAL or LSM tree.
    - hawski 3y ago
      Most programs either write a full file every time and replace the old file by a single move or append to an old file. Writting in the middle could happen in a program writting to some kind of archive or disk image. There is probably a whole group of programs that do this I'm not familiar with, but I'm pretty sure of my first sentence.
      - 3y ago
MontyCarloHall 3y ago
I’ve experimented with using gcsfuse and its AWS equivalent, s3fs-fuse in production. At best, they are suited to niche applications; at worst, they are merely nice toys. The issue is that every file system operation is fundamentally an HTTP request, so the latency is several orders of magnitude higher than the equivalent disk operation. For certain applications that consistently read limited subsets of the filesystem, this can be mitigated somewhat by the disk cache, but for applications that would thrash the cache, cloud buckets are simply not a good storage backend if you desire disk-like access. What I would really like to see is a two-tier cache system: most recently accessed files are cached to RAM, with less recently accessed files spilling over to a disk-backed cache. That would open up a world of additional applications whose useful cache size exceeds practical RAM amounts.
- qsort 3y ago
  My personal conspiracy theory: most "cloud services" are just... bad. VMs and disk space I understand completely, having machines on-prem is too much of an hassle and the price isn't that bad. But for stuff like this, managed services, databases especially, you're just getting scammed.
- tyingq 3y ago
  Well, and there's no such thing as opening a file and modifying some small part of it. That's emulated with a full rewrite of the whole object.
- linsomniac 3y ago
  >What I would really like to see is a two-tier cache system Is there any sort of Linux HSM (Hieracrhical Storage Manager)? I haven't see any and have been a bit surprised nothing has really developed there. They can manage putting hot data in RAM, SSDs, colder or larger data on spinning rust, deep freezing onto a tape silo or a cloud storage... Some of the NAS devices and RAID cards can support a two-tier caching or data migration using SSDs, where hot or highly-random data (usually identified by smaller write sizes) go to the SSDs, and then can migrate to the spinning discs. I've done some "poor mans" version of this using LVM, where I can "pvmove" blocks of a logical volume between spinning discs and SSDs, which is pretty slick, but a very crude tool.
askvictor 3y ago
Now for official Google Drive support on Linux...
- curt15 3y ago
  How do Googlers access Google Drive from their Linux workstations? Do they have an internal GDrive client?
  - martius 3y ago
    Not that I know of, we have some virtual filesystems for specific things, but in general Drive is for shared docs, videos (recorded meetings/presentations) and things like this. We don't use drive to store other files. Actually, we don't really "store files" since almost everything we need is remote. See for instance this discussion: https://news.ycombinator.com/item?id=13561096 https://news.ycombinator.com/item?id=13561096
  - plaidfuji 3y ago
    I wonder the same, but I also wonder what the actual use case is for the Drive app on Linux. For me, Drive is mostly for syncing office docs (namely MS-office docs), PDFs and images among teams. That type of work doesn’t lend itself well to a Linux env anyway. And for programming-heavy sync tasks, a user will more likely use a remote Git repo for code and GCS for data. Does google even use MS office internally?
    - capableweb 3y ago
      > Does google even use MS office internally That'd be weird, considering they have their own suite of office tools. Kind of like if Microsoft would be using Google Cloud rather than Azure internally.
      - olyjohn 3y ago
        Kinda like if Microsoft ran Linux on their cloud...
manigandham 3y ago
Object storage is a higher-level abstraction than block-storage. FUSE and similar tech can do the job for basic requirements like read-only access by legacy applications but rarely works well for other scenarios. A more complex layer like https://objectivefs.com/ https://objectivefs.com/ (based on the S3 API) would be more useful, although I would've expected the cloud providers to scale their own block-store/SANs backed with object-stores by now.
goodpoint 3y ago
FUSE is really not suitable for this.
ISL 3y ago
Can this be used to mount Drive under linux?
- lern_too_spel 3y ago
  Use rclone.
- hobo_mark 3y ago
  GCS is not Drive, it's Google's equivalent of S3.
- speedster217 3y ago
  https://github.com/astrada/google-drive-ocamlfuse https://github.com/astrada/google-drive-ocamlfuse is one option
remram 3y ago
See also: JuiceFS: https://juicefs.com/ https://juicefs.com/ Adds a DBMS or key-value store for metadata, making the filesystem much faster (POSIX, small overwrites don't have to replace a full object in the GCS/S3 backend). Almost certainly a better solution if you want to turn your object storage into a mountable filesystem, with the (big) caveat that you can't access the files directly in the bucket (they are not stored transparently).
- e1g 3y ago
  JuiceFS is mostly POSIX compatible, but there are important caveats such as no extended ACL, copying files changes their mtime (impacts backup tools), it offers "close-to-open" consistency (dangerous for log appenders), etc. Choosing an appropriate solution in this space still depends on what you need to do with the storage, and a few other options are MooseFS (https://github.com/moosefs/moosefs https://github.com/moosefs/moosefs) SeaweedFS (https://github.com/seaweedfs/seaweedfs https://github.com/seaweedfs/seaweedfs) Curve (https://github.com/opencurve/curve https://github.com/opencurve/curve) GeeseFS (https://github.com/yandex-cloud/geesefs https://github.com/yandex-cloud/geesefs)
  - remram 3y ago
    Those other ones can't use GCS though, which is why I pointed to JuiceFS (at least the ones I recognize).
jijji 3y ago
I've been using rclone [0] to do the same under linux for years, how is this different? [0] https://rclone.org https://rclone.org
- dallbee 3y ago
  Unfortunately it's common to have a policy in place disallowing 3rd-party app api access to drive storage. This prevents apps like rclone from working, but the drive client works because it isn't 3rd-party.
iamjk 3y ago
I mean I get why everyone wants everything to be fuse-compatible but some things just aren't meant to be done.
- ggambetta 3y ago
  "Your scientists were so preoccupied with whether or not they could, they didn't stop to think if they should!"
nickcw 3y ago
As the author of rclone I thought I'd have a quick look through the docs to see what this is about. From reading the docs, it looks very similar to `rclone mount` with `--vfs-cache-mode off` (the default). The limitations are almost identical. * Metadata: Cloud Storage FUSE does not transfer object metadata when uploading files to Cloud Storage, with the exception of mtime and symlink targets. This means that you cannot set object metadata when you upload files using Cloud Storage FUSE. If you need to preserve object metadata, consider uploading files using gsutil, the JSON API, or the Google Cloud console. * Concurrency: Cloud Storage FUSE does not provide concurrency control for multiple writes to the same file. When multiple writes try to replace a file, the last write wins and all previous writes are lost. There is no merging, version control, or user notification of the subsequent overwrite. * Linking: Cloud Storage FUSE does not support hard links. * File locking and file patching: Cloud Storage FUSE does not support file locking or file patching. As such, you should not store version control system repositories in Cloud Storage FUSE mount points, as version control systems rely on file locking and patching. Additionally, you should not use Cloud Storage FUSE as a filer replacement. * Semantics: Semantics in Cloud Storage FUSE are different from semantics in a traditional file system. For example, metadata like last access time are not supported, and some metadata operations like directory renaming are not atomic. For a list of differences between Cloud Storage FUSE semantics and traditional file system semantics, see Semantics in the Cloud Storage FUSE GitHub documentation. * Overwriting in the middle of a file: Cloud Storage FUSE does not support overwriting in the middle of a file. Only sequential writes are supported. Access: Authorization for files is governed by Cloud Storage permissions. POSIX-style access control does not work. However rclone has `--vfs-cache-mode writes` which caches file writes to disk first to allow overwriting in the middle of a file and `--vfs-cache-mode full` to cache all objects on a LRU basis. They both make the file system a whole lot more POSIX compatible and most applications will run using `--vfs-cache-mode writes` unlike `--vfs-cache-mode off`. And of course rclone supports s3/azureblob/b2/r2/sftp/webdav/etc/etc also... I don't think it is possible to adapt something with cloud storage semantics to a file system without caching to disk, unless you are willing to leave behind the 1:1 mapping of files seen in the mount to object in the cloud storage.
retrocryptid 3y ago
This has been a thing for a while; I remember using it (or something like it) several years ago. While it's great for random files you might want to place in the G-Cloud, what I really wanted was to access my google docs content from the Linux command line. And you can do that, it's just that they're in non-obvious, non-documented, frequently changing formats that will only ever be usable with Google Docs. But if you're using the google cloud like you might use Box.Net or DropBox, it seems fine for light usage.
milesward 3y ago
Please, listen to me: use this only in extremely limited cases where performance, stability, and cost efficiency are not paramount. An object store is not a file system no matter how hard you bludgeon it.
ofek 3y ago
I do appreciate that Google is now officially supporting gcsfuse because it genuinely is a great project. However, their Kubernetes CSI driver seems to have in large part copied code from the one I and a co-maintainer have been working on for years: - https://github.com/GoogleCloudPlatform/gcs-fuse-csi-driver https://github.com/GoogleCloudPlatform/gcs-fuse-csi-driver - https://github.com/ofek/csi-gcs https://github.com/ofek/csi-gcs Here is the initial commit: https://github.com/GoogleCloudPlatform/gcs-fuse-csi-driver/commit/04df4c7f0940a41530e94342e26ac2e65ef8cb77 https://github.com/GoogleCloudPlatform/gcs-fuse-csi-driver/c... Notice for example not just the code but also the associated files. In the Dockerfile it blatantly copied the one from my repo, even the dual license I chose because I was very into Rust at the time. Or take a look at the deployment examples which use Kustomize which I like but is very uncommon and most Kubernetes projects provide Helm charts instead. They were most certainly aware of the project because Google reached out to discuss potential collaboration but never responded back: https://imgur.com/a/KDuf9mj https://imgur.com/a/KDuf9mj
- ofek 3y ago
  Update: attribution has been added to the readme file https://github.com/GoogleCloudPlatform/gcs-fuse-csi-driver/commit/40dbfc5b736948f82051539ed9926609f399b792 https://github.com/GoogleCloudPlatform/gcs-fuse-csi-driver/c...
- js2023 3y ago
  Hi Ofek, I am a contributor who works on the Google Cloud Storage FUSE CSI Driver project. The project is partially inspired by your CSI implementation. Thank you so much for the contribution to the Kubernetes community. However, I would like to clarify a few things regarding your post. The Cloud Storage FUSE CSI Driver project does not have “in large part copied code” from your implementation. The initial commit you referred to in the post was based on a fork of another open source project: https://github.com/kubernetes-sigs/gcp-filestore-csi-driver https://github.com/kubernetes-sigs/gcp-filestore-csi-driver. If you compare the Google Cloud Storage FUSE CSI Driver repo with the Google Cloud Filestore CSI Driver repo, you will notice the obvious similarities, in terms of the code structure, the Dockerfile, the usage of Kustomize, and the way the CSI is implemented. Moreover, the design of the Google Cloud Storage FUSE CSI Driver included a proxy server, and then evolved to a sidecar container mode, which are all significantly different from your implementation. As for the Dockerfile annotations you pointed out in the initial commit, I did follow the pattern in your repo because I thought it was the standard way to declare the copyright. However, it didn't take me too long to realize that the Dockerfile annotations are not required, so I removed them. Thank you again for your contribution to the open source community. I have included your project link on the readme page. I take the copyright very seriously, so please feel free to directly create issues or PRs on the Cloud Storage FUSE CSI Driver GitHub project page if I missed any other copyright information.

149 points by mvolfik 3y ago

Is this the same gcsfuse that's been around for years, only now with official Google support? https://github.com/GoogleCloudPlatform/gcsfuse https://github.com/GoogleCloudPlatform/gcsfuse

beastman82 3y ago
yes > export GCSFUSE_REPO=gcsfuse-`lsb_release -c -s`
scoobydoobydrew 3y ago
Looking at change descriptions, looks like underlying changes were made to get to this like now using GO client library. I would expect a more stable product, and better performance which looks like the performance benchmarks located under docs has been updated as well. Happy to finally see Google standing behind this, and the official CSI driver is really cool to see.
throwdbaaway 3y ago
Heh, my old laptop has a git clone of this from September 1st 2016.

trollied 3y ago

Be aware that this is not free: "Cloud Storage FUSE is available free of charge, but the storage, metadata, and network I/O it generates to and from Cloud Storage are charged like any other Cloud Storage interface. In other words, all data transfer and operations performed by Cloud Storage FUSE map to Cloud Storage transfers and operations, and are charged accordingly."

rippercushions 3y ago
Using FUSE doesn't cost you anything extra, but it doesn't make the underlying storage free.
- ElectricalUnion 3y ago
  You will be doing storage operations silently and in a unoptimized fashion, more so if the underlying FUSE filesystem is implemented in a naive fashion. For example, Cloud Storage never moves or renames your objects; copying and deleting the original one instead. This can end up costing quite a lot if you're using data other that in "standard store" because of minimum storage duration.

jefftk 3y ago

Cloud Storage FUSE does not support overwriting in the middle of a file. Only sequential writes are supported. This seems like a big limitation?

throwawaaarrgh 3y ago
pretty standard limitation of object storage services iirc
- jefftk 3y ago
  Doesn't this mean that most programs you might want to use with the FUSE API won't actually work? They'll do fine for a while, until they try to seek, and then they'll get an error? Or is there a large group of programs that only ever write sequentially?
  - jsnell 3y ago
    I'd think non-appending writes are quite rare in practice, other than databases. Even when the application is logically overwriting data, in other kinds of programs it's almost always implemented as writing to a new file + an atomic rename, not in-place modification.
    - dontlaugh 3y ago
      Even databases tend to do sequential writes, whether to a WAL or LSM tree.
  - hawski 3y ago
    Most programs either write a full file every time and replace the old file by a single move or append to an old file. Writting in the middle could happen in a program writting to some kind of archive or disk image. There is probably a whole group of programs that do this I'm not familiar with, but I'm pretty sure of my first sentence.
    - 3y ago

MontyCarloHall 3y ago

I’ve experimented with using gcsfuse and its AWS equivalent, s3fs-fuse in production. At best, they are suited to niche applications; at worst, they are merely nice toys. The issue is that every file system operation is fundamentally an HTTP request, so the latency is several orders of magnitude higher than the equivalent disk operation. For certain applications that consistently read limited subsets of the filesystem, this can be mitigated somewhat by the disk cache, but for applications that would thrash the cache, cloud buckets are simply not a good storage backend if you desire disk-like access. What I would really like to see is a two-tier cache system: most recently accessed files are cached to RAM, with less recently accessed files spilling over to a disk-backed cache. That would open up a world of additional applications whose useful cache size exceeds practical RAM amounts.

qsort 3y ago
My personal conspiracy theory: most "cloud services" are just... bad. VMs and disk space I understand completely, having machines on-prem is too much of an hassle and the price isn't that bad. But for stuff like this, managed services, databases especially, you're just getting scammed.
tyingq 3y ago
Well, and there's no such thing as opening a file and modifying some small part of it. That's emulated with a full rewrite of the whole object.
linsomniac 3y ago
>What I would really like to see is a two-tier cache system Is there any sort of Linux HSM (Hieracrhical Storage Manager)? I haven't see any and have been a bit surprised nothing has really developed there. They can manage putting hot data in RAM, SSDs, colder or larger data on spinning rust, deep freezing onto a tape silo or a cloud storage... Some of the NAS devices and RAID cards can support a two-tier caching or data migration using SSDs, where hot or highly-random data (usually identified by smaller write sizes) go to the SSDs, and then can migrate to the spinning discs. I've done some "poor mans" version of this using LVM, where I can "pvmove" blocks of a logical volume between spinning discs and SSDs, which is pretty slick, but a very crude tool.

askvictor 3y ago

Now for official Google Drive support on Linux...

curt15 3y ago
How do Googlers access Google Drive from their Linux workstations? Do they have an internal GDrive client?
- martius 3y ago
  Not that I know of, we have some virtual filesystems for specific things, but in general Drive is for shared docs, videos (recorded meetings/presentations) and things like this. We don't use drive to store other files. Actually, we don't really "store files" since almost everything we need is remote. See for instance this discussion: https://news.ycombinator.com/item?id=13561096 https://news.ycombinator.com/item?id=13561096
- plaidfuji 3y ago
  I wonder the same, but I also wonder what the actual use case is for the Drive app on Linux. For me, Drive is mostly for syncing office docs (namely MS-office docs), PDFs and images among teams. That type of work doesn’t lend itself well to a Linux env anyway. And for programming-heavy sync tasks, a user will more likely use a remote Git repo for code and GCS for data. Does google even use MS office internally?
  - capableweb 3y ago
    > Does google even use MS office internally That'd be weird, considering they have their own suite of office tools. Kind of like if Microsoft would be using Google Cloud rather than Azure internally.
    - olyjohn 3y ago
      Kinda like if Microsoft ran Linux on their cloud...

manigandham 3y ago

Object storage is a higher-level abstraction than block-storage. FUSE and similar tech can do the job for basic requirements like read-only access by legacy applications but rarely works well for other scenarios. A more complex layer like https://objectivefs.com/ https://objectivefs.com/ (based on the S3 API) would be more useful, although I would've expected the cloud providers to scale their own block-store/SANs backed with object-stores by now.

goodpoint 3y ago

FUSE is really not suitable for this.

ISL 3y ago

Can this be used to mount Drive under linux?

lern_too_spel 3y ago
Use rclone.
hobo_mark 3y ago
GCS is not Drive, it's Google's equivalent of S3.
speedster217 3y ago
https://github.com/astrada/google-drive-ocamlfuse https://github.com/astrada/google-drive-ocamlfuse is one option

remram 3y ago

See also: JuiceFS: https://juicefs.com/ https://juicefs.com/ Adds a DBMS or key-value store for metadata, making the filesystem much faster (POSIX, small overwrites don't have to replace a full object in the GCS/S3 backend). Almost certainly a better solution if you want to turn your object storage into a mountable filesystem, with the (big) caveat that you can't access the files directly in the bucket (they are not stored transparently).

e1g 3y ago
JuiceFS is mostly POSIX compatible, but there are important caveats such as no extended ACL, copying files changes their mtime (impacts backup tools), it offers "close-to-open" consistency (dangerous for log appenders), etc. Choosing an appropriate solution in this space still depends on what you need to do with the storage, and a few other options are MooseFS (https://github.com/moosefs/moosefs https://github.com/moosefs/moosefs) SeaweedFS (https://github.com/seaweedfs/seaweedfs https://github.com/seaweedfs/seaweedfs) Curve (https://github.com/opencurve/curve https://github.com/opencurve/curve) GeeseFS (https://github.com/yandex-cloud/geesefs https://github.com/yandex-cloud/geesefs)
- remram 3y ago
  Those other ones can't use GCS though, which is why I pointed to JuiceFS (at least the ones I recognize).

jijji 3y ago

I've been using rclone [0] to do the same under linux for years, how is this different? [0] https://rclone.org https://rclone.org

dallbee 3y ago
Unfortunately it's common to have a policy in place disallowing 3rd-party app api access to drive storage. This prevents apps like rclone from working, but the drive client works because it isn't 3rd-party.

iamjk 3y ago

I mean I get why everyone wants everything to be fuse-compatible but some things just aren't meant to be done.

ggambetta 3y ago
"Your scientists were so preoccupied with whether or not they could, they didn't stop to think if they should!"

nickcw 3y ago

As the author of rclone I thought I'd have a quick look through the docs to see what this is about. From reading the docs, it looks very similar to `rclone mount` with `--vfs-cache-mode off` (the default). The limitations are almost identical. * Metadata: Cloud Storage FUSE does not transfer object metadata when uploading files to Cloud Storage, with the exception of mtime and symlink targets. This means that you cannot set object metadata when you upload files using Cloud Storage FUSE. If you need to preserve object metadata, consider uploading files using gsutil, the JSON API, or the Google Cloud console. * Concurrency: Cloud Storage FUSE does not provide concurrency control for multiple writes to the same file. When multiple writes try to replace a file, the last write wins and all previous writes are lost. There is no merging, version control, or user notification of the subsequent overwrite. * Linking: Cloud Storage FUSE does not support hard links. * File locking and file patching: Cloud Storage FUSE does not support file locking or file patching. As such, you should not store version control system repositories in Cloud Storage FUSE mount points, as version control systems rely on file locking and patching. Additionally, you should not use Cloud Storage FUSE as a filer replacement. * Semantics: Semantics in Cloud Storage FUSE are different from semantics in a traditional file system. For example, metadata like last access time are not supported, and some metadata operations like directory renaming are not atomic. For a list of differences between Cloud Storage FUSE semantics and traditional file system semantics, see Semantics in the Cloud Storage FUSE GitHub documentation. * Overwriting in the middle of a file: Cloud Storage FUSE does not support overwriting in the middle of a file. Only sequential writes are supported. Access: Authorization for files is governed by Cloud Storage permissions. POSIX-style access control does not work. However rclone has `--vfs-cache-mode writes` which caches file writes to disk first to allow overwriting in the middle of a file and `--vfs-cache-mode full` to cache all objects on a LRU basis. They both make the file system a whole lot more POSIX compatible and most applications will run using `--vfs-cache-mode writes` unlike `--vfs-cache-mode off`. And of course rclone supports s3/azureblob/b2/r2/sftp/webdav/etc/etc also... I don't think it is possible to adapt something with cloud storage semantics to a file system without caching to disk, unless you are willing to leave behind the 1:1 mapping of files seen in the mount to object in the cloud storage.

retrocryptid 3y ago

This has been a thing for a while; I remember using it (or something like it) several years ago. While it's great for random files you might want to place in the G-Cloud, what I really wanted was to access my google docs content from the Linux command line. And you can do that, it's just that they're in non-obvious, non-documented, frequently changing formats that will only ever be usable with Google Docs. But if you're using the google cloud like you might use Box.Net or DropBox, it seems fine for light usage.

milesward 3y ago

Please, listen to me: use this only in extremely limited cases where performance, stability, and cost efficiency are not paramount. An object store is not a file system no matter how hard you bludgeon it.

ofek 3y ago

I do appreciate that Google is now officially supporting gcsfuse because it genuinely is a great project. However, their Kubernetes CSI driver seems to have in large part copied code from the one I and a co-maintainer have been working on for years: - https://github.com/GoogleCloudPlatform/gcs-fuse-csi-driver https://github.com/GoogleCloudPlatform/gcs-fuse-csi-driver - https://github.com/ofek/csi-gcs https://github.com/ofek/csi-gcs Here is the initial commit: https://github.com/GoogleCloudPlatform/gcs-fuse-csi-driver/commit/04df4c7f0940a41530e94342e26ac2e65ef8cb77 https://github.com/GoogleCloudPlatform/gcs-fuse-csi-driver/c... Notice for example not just the code but also the associated files. In the Dockerfile it blatantly copied the one from my repo, even the dual license I chose because I was very into Rust at the time. Or take a look at the deployment examples which use Kustomize which I like but is very uncommon and most Kubernetes projects provide Helm charts instead. They were most certainly aware of the project because Google reached out to discuss potential collaboration but never responded back: https://imgur.com/a/KDuf9mj https://imgur.com/a/KDuf9mj

ofek 3y ago
Update: attribution has been added to the readme file https://github.com/GoogleCloudPlatform/gcs-fuse-csi-driver/commit/40dbfc5b736948f82051539ed9926609f399b792 https://github.com/GoogleCloudPlatform/gcs-fuse-csi-driver/c...
js2023 3y ago
Hi Ofek, I am a contributor who works on the Google Cloud Storage FUSE CSI Driver project. The project is partially inspired by your CSI implementation. Thank you so much for the contribution to the Kubernetes community. However, I would like to clarify a few things regarding your post. The Cloud Storage FUSE CSI Driver project does not have “in large part copied code” from your implementation. The initial commit you referred to in the post was based on a fork of another open source project: https://github.com/kubernetes-sigs/gcp-filestore-csi-driver https://github.com/kubernetes-sigs/gcp-filestore-csi-driver. If you compare the Google Cloud Storage FUSE CSI Driver repo with the Google Cloud Filestore CSI Driver repo, you will notice the obvious similarities, in terms of the code structure, the Dockerfile, the usage of Kustomize, and the way the CSI is implemented. Moreover, the design of the Google Cloud Storage FUSE CSI Driver included a proxy server, and then evolved to a sidecar container mode, which are all significantly different from your implementation. As for the Dockerfile annotations you pointed out in the initial commit, I did follow the pattern in your repo because I thought it was the standard way to declare the copyright. However, it didn't take me too long to realize that the Dockerfile annotations are not required, so I removed them. Thank you again for your contribution to the open source community. I have included your project link on the readme page. I take the copyright very seriously, so please feel free to directly create issues or PRs on the Cloud Storage FUSE CSI Driver GitHub project page if I missed any other copyright information.