Sharing local files with Kubernetes clusters
(2021) In LU-CS-EX EDAM05 20211Department of Computer Science
- Abstract
- Data science development largely revolves around working in notebooks, and
these usually run in a cluster environment. These cloud notebooks do however come with limitations. Support for tools like version control, testing, refactoring and linting is limited, which negatively impacts developer experience and software quality.
We propose a networked file system for sharing local files and directories with a remote cloud server. This allows the developer to work in a cloud notebook without having to upload files to an external storage provider, and enables the use of traditional development tools since all files are stored on the local machine. The networked file system is implemented as a FUSE file system and is made available to... (More) - Data science development largely revolves around working in notebooks, and
these usually run in a cluster environment. These cloud notebooks do however come with limitations. Support for tools like version control, testing, refactoring and linting is limited, which negatively impacts developer experience and software quality.
We propose a networked file system for sharing local files and directories with a remote cloud server. This allows the developer to work in a cloud notebook without having to upload files to an external storage provider, and enables the use of traditional development tools since all files are stored on the local machine. The networked file system is implemented as a FUSE file system and is made available to Kubernetes as a custom volume driver.
We show that it is possible to build a widely compatible file sharing solution for any Kubernetes workload. It is easy to use and has reasonable performance. (Less) - Popular Abstract (Swedish)
- I takt med att världen blir mer datadriven flyttas allt fler stora beräkningar till molnet. Då tillkommer ofta problem med filhantering. Detta examensarbete bidrar med ett nytt sätt att dela filer med molnet, där molntjänsten får tillgång till filer som lagras på användarens dator.
Please use this url to cite or link to this publication:
http://lup.lub.lu.se/student-papers/record/9066685
- author
- Jakobsson, Martin LU and Henriksson, Johan LU
- supervisor
-
- Markus Borg LU
- organization
- course
- EDAM05 20211
- year
- 2021
- type
- H2 - Master's Degree (Two Years)
- subject
- keywords
- file system, fuse, kubernetes, docker, notebook, container, cowait
- publication/series
- LU-CS-EX
- report number
- 2021-37
- ISSN
- 1650-2884
- language
- English
- id
- 9066685
- date added to LUP
- 2021-10-11 09:16:05
- date last changed
- 2021-11-03 17:33:32
@misc{9066685, abstract = {{Data science development largely revolves around working in notebooks, and these usually run in a cluster environment. These cloud notebooks do however come with limitations. Support for tools like version control, testing, refactoring and linting is limited, which negatively impacts developer experience and software quality. We propose a networked file system for sharing local files and directories with a remote cloud server. This allows the developer to work in a cloud notebook without having to upload files to an external storage provider, and enables the use of traditional development tools since all files are stored on the local machine. The networked file system is implemented as a FUSE file system and is made available to Kubernetes as a custom volume driver. We show that it is possible to build a widely compatible file sharing solution for any Kubernetes workload. It is easy to use and has reasonable performance.}}, author = {{Jakobsson, Martin and Henriksson, Johan}}, issn = {{1650-2884}}, language = {{eng}}, note = {{Student Paper}}, series = {{LU-CS-EX}}, title = {{Sharing local files with Kubernetes clusters}}, year = {{2021}}, }