Initial commit: OCA Storage packages (17 packages)

odoo-bringout-oca-storage-fs_storage/fs_storage/readme/CONTRIBUTORS.rst

@@ -0,0 +1,2 @@
+* Laurent Mignon <laurent.mignon@acsone.eu>
+* Sébastien BEAU <sebastien.beau@akretion.com>

odoo-bringout-oca-storage-fs_storage/fs_storage/readme/DESCRIPTION.rst

@@ -0,0 +1,63 @@
+This addon is a technical addon that allows you to define filesystem like
+storage for your data. It's used by other addons to store their data in a
+transparent way into different kind of storages.
+
+Through the fs.storage record, you get access to an object that implements
+the `fsspec.spec.AbstractFileSystem <https://filesystem-spec.readthedocs.io/en/
+latest/api.html#fsspec.spec.AbstractFileSystem>`_ interface and therefore give
+you an unified interface to access your data whatever the storage protocol you
+decide to use.
+
+The list of supported protocols depends on the installed fsspec implementations.
+By default, the addon will install the following protocols:
+
+* LocalFileSystem
+* MemoryFileSystem
+* ZipFileSystem
+* TarFileSystem
+* FTPFileSystem
+* CachingFileSystem
+* WholeFileSystem
+* SimplCacheFileSystem
+* ReferenceFileSystem
+* GenericFileSystem
+* DirFileSystem
+* DatabricksFileSystem
+* GitHubFileSystem
+* JupiterFileSystem
+* OdooFileSystem
+
+The OdooFileSystem is the one that allows you to store your data into a directory
+mounted into your Odoo's storage directory. This is the default FS Storage
+when creating a new fs.storage record.
+
+Others protocols are available through the installation of additional
+python packages:
+
+* DropboxDriveFileSystem -> `pip install fsspec[dropbox]`
+* HTTPFileSystem -> `pip install fsspec[http]`
+* HTTPSFileSystem -> `pip install fsspec[http]`
+* GCSFileSystem -> `pip install fsspec[gcs]`
+* GSFileSystem -> `pip install fsspec[gs]`
+* GoogleDriveFileSystem -> `pip install gdrivefs`
+* SFTPFileSystem -> `pip install fsspec[sftp]`
+* HaddoopFileSystem -> `pip install fsspec[hdfs]`
+* S3FileSystem -> `pip install fsspec[s3]`
+* WandbFS -> `pip install wandbfs`
+* OCIFileSystem -> `pip install fsspec[oci]`
+* AsyncLocalFileSystem -> `pip install 'morefs[asynclocalfs]`
+* AzureDatalakeFileSystem -> `pip install fsspec[adl]`
+* AzureBlobFileSystem -> `pip install fsspec[abfs]`
+* DaskWorkerFileSystem -> `pip install fsspec[dask]`
+* GitFileSystem -> `pip install fsspec[git]`
+* SMBFileSystem -> `pip install fsspec[smb]`
+* LibArchiveFileSystem -> `pip install fsspec[libarchive]`
+* OSSFileSystem -> `pip install ossfs`
+* WebdavFileSystem -> `pip install webdav4`
+* DVCFileSystem -> `pip install dvc`
+* XRootDFileSystem -> `pip install fsspec-xrootd`
+
+This list of supported protocols is not exhaustive or could change in the future
+depending on the fsspec releases. You can find more information about the
+supported protocols on the `fsspec documentation
+<https://filesystem-spec.readthedocs.io/en/latest/api.html#fsspec.spec.AbstractFileSystem>`_.

odoo-bringout-oca-storage-fs_storage/fs_storage/readme/HISTORY.rst

@@ -0,0 +1,33 @@
+16.0.1.2.0 (2024-02-06)
+~~~~~~~~~~~~~~~~~~~~~~~
+
+**Features**
+
+- Invalidate FS filesystem object cache when the connection fails, forcing a reconnection. (`#320 <https://github.com/OCA/storage/issues/320>`_)
+
+
+16.0.1.1.0 (2023-12-22)
+~~~~~~~~~~~~~~~~~~~~~~~
+
+**Features**
+
+- Add parameter on storage backend to resolve protocol options values starting with $ from environment variables (`#303 <https://github.com/OCA/storage/issues/303>`_)
+
+
+16.0.1.0.3 (2023-10-17)
+~~~~~~~~~~~~~~~~~~~~~~~
+
+**Bugfixes**
+
+- Fix access to technical models to be able to upload attachments for users with basic access (`#289 <https://github.com/OCA/storage/issues/289>`_)
+
+
+16.0.1.0.2 (2023-10-09)
+~~~~~~~~~~~~~~~~~~~~~~~
+
+**Bugfixes**
+
+- Avoid config error when using the webdav protocol. The auth option is expected
+  to be a tuple not a list. Since our config is loaded from a json file, we
+  cannot use tuples. The fix converts the list to a tuple when the config is
+  related to a webdav protocol and the auth option is into the confix. (`#285 <https://github.com/OCA/storage/issues/285>`_)

odoo-bringout-oca-storage-fs_storage/fs_storage/readme/ROADMAP.rst

@@ -0,0 +1,9 @@
+* Transactions: fsspec comes with a transactional mechanism that once started,
+  gathers all the files created during the transaction, and if the transaction
+  is committed, moves them to their final locations. It would be useful to
+  bridge this with the transactional mechanism of odoo. This would allow to
+  ensure that all the files created during a transaction are either all
+  moved to their final locations, or all deleted if the transaction is rolled
+  back. This mechanism is only valid for files created during the transaction
+  by a call to the `open` method of the file system. It is not valid for others
+  operations, such as `rm`, `mv_file`, ... .

odoo-bringout-oca-storage-fs_storage/fs_storage/readme/USAGE.rst

@@ -0,0 +1,94 @@
+Configuration
+~~~~~~~~~~~~~
+
+When you create a new backend, you must specify the following:
+
+* The name of the backend. This is the name that will be used to
+  identify the backend into Odoo
+* The code of the backend. This code will identify the backend into the store_fname
+  field of the ir.attachment model. This code must be unique. It will be used
+  as scheme. example of the store_fname field: ``odoofs://abs34Tg11``.
+* The protocol used by the backend. The protocol refers to the supported
+  protocols of the fsspec python package.
+* A directory path. This is a root directory from which the filesystem will
+  be mounted. This directory must exist.
+* The protocol options. These are the options that will be passed to the
+  fsspec python package when creating the filesystem. These options depend
+  on the protocol used and are described in the fsspec documentation.
+* Resolve env vars. This options resolves the protocol options values starting
+  with $ from environment variables
+* Check Connection Method. If set, Odoo will always check the connection before using
+  a storage and it will remove the fs connection from the cache if the check fails.
+
+  * ``Create Marker file`` : create a hidden file on remote and then check it exists with
+    Use it if you have write access to the remote and if it is not an issue to leave
+    the marker file in the root directory.
+  * ``List file`` : list all files from the root directory. You can use it if the directory
+    path does not contain a big list of files (for performance reasons)
+
+Some protocols defined in the fsspec package are wrappers around other
+protocols. For example, the SimpleCacheFileSystem protocol is a wrapper
+around any local filesystem protocol. In such cases, you must specify into the
+protocol options the protocol to be wrapped and the options to be passed to
+the wrapped protocol.
+
+For example, if you want to create a backend that uses the SimpleCacheFileSystem
+protocol, after selecting the SimpleCacheFileSystem protocol, you must specify
+the protocol options as follows:
+
+.. code-block:: python
+
+    {
+        "directory_path": "/tmp/my_backend",
+        "target_protocol": "odoofs",
+        "target_options": {...},
+    }
+
+In this example, the SimpleCacheFileSystem protocol will be used as a wrapper
+around the odoofs protocol.
+
+Server Environment
+~~~~~~~~~~~~~~~~~~
+
+To ease the management of the filesystem storages configuration accross the different
+environments, the configuration of the filesystem storages can be defined in
+environment files or directly in the main configuration file. For example, the
+configuration of a filesystem storage with the code `fsprod` can be provided in the
+main configuration file as follows:
+
+.. code-block:: ini
+
+  [fs_storage.fsprod]
+  protocol=s3
+  options={"endpoint_url": "https://my_s3_server/", "key": "KEY", "secret": "SECRET"}
+  directory_path=my_bucket
+
+To work, a `storage.backend` record must exist with the code `fsprod` into the database.
+In your configuration section, you can specify the value for the following fields:
+
+* `protocol`
+* `options`
+* `directory_path`
+
+Migration from storage_backend
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The fs_storage addon can be used to replace the storage_backend addon. (It has
+been designed to be a drop-in replacement for the storage_backend addon). To
+ease the migration, the `fs.storage` model defines the high-level methods
+available in the storage_backend model. These methods are:
+
+* `add`
+* `get`
+* `list_files`
+* `find_files`
+* `move_files`
+* `delete`
+
+These methods are wrappers around the methods of the `fsspec.AbstractFileSystem`
+class (see https://filesystem-spec.readthedocs.io/en/latest/api.html#fsspec.spec.AbstractFileSystem).
+These methods are marked as deprecated and will be removed in a future version (V18)
+of the addon. You should use the methods of the `fsspec.AbstractFileSystem` class
+instead since they are more flexible and powerful. You can access the instance
+of the `fsspec.AbstractFileSystem` class using the `fs` property of a `fs.storage`
+record.