2 ms·
i have had problems integrating apache spark using google storage. especially because s3 is directly supported in spark. if you are api compatible with s3, cou
by sandGorgon 10y ago
i have had problems integrating apache spark using google storage. especially because s3 is directly supported in spark.
if you are api compatible with s3, could you make it easy /possible to work with google storage inside spark?
remember i may or may not run my spark on Dataproc.
- bluedonuts 10y agoYou can use the Google cloud storage connector (https://cloud.google.com/hadoop/google-cloud-storage-connector https://cloud.google.com/hadoop/google-cloud-storage-connect...) which works with hadoop (and therefore spark).