Added DMS API to do list in docs

This commit is contained in:
shb 2025-06-21 08:57:08 +08:00
parent 071b1c4529
commit 600e9364f9

View File

@ -0,0 +1,65 @@
# DMS API fixes
## Folder operations
Folder operations were added to directly reflect folder structures in the database. Preparing folder structures in AWS S3 directly incurs extra access charges and cannot be done without altering the data in the storage (aka more money loss).
Please refer to the (dms-api.md)[dms-api.md] file for details on how to perform CRUD operations for folders.
**Note:** All folder CRUDs are done using database, so `.env` for database must be intialized first.
## AWS S3 Upload configuration
Uploading to S3 utilizes direct-to-storage uploads. To understand how this works, please read the following steps on how the upload works:
1. When uploading a file, the frontend sends file name and file type **only** to the backend. **DO NOT** attempt to send the entire file as `multipart/form-data` to the backend API as it is forbidden for some reason. **No** it is not caused by CORS. **Yes I have tried other methods and all of them don't work on sending the file to the backend**.
2. The backend then sends the file name and file type to AWS S3 Request Presigner and gets a temporary presigned URL to the AWS S3 Bucket.
3. The backend sends the presigned URL link to the frontend.
4. The frontend then uploads the file to the presigned URL directly.
**Note**: The file does not get sent to the backend. Instead, the frontend sends the file directly to S3 using a presigned URL obtained from the backend, hence the term "direct-to-storage".
**Caution**
AWS S3 Buckets have their own CORS configurations. If you get a `403 Forbidden` response from AWS in the browser console, then CORS must be (manually set up in the S3 Bucket settings)[https://docs.aws.amazon.com/AmazonS3/latest/userguide/ManageCorsUsing.html]. **Note:** This requires access to an AWS console, which needs an account to run. It is recommended to use an (IAM user role)[https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_create.html] to configure these features for better security against attacks.
When configured correctly, the file should be uploaded to a folder named `upload/` in the S3 Bucket.
# DMS API To Do List
1. **Create a GET route for obtaining and downloading files from S3**
- There is no API function yet to get all the files in S3 and offer file download functionality.
- How to show the correct file in a directory without having to perform GET requests to AWS constantly (every call to AWS has costs, so avoid accessing it as much as possible).
2. **Create DELETE route for deleting files from S3**
- There is also no DELETE function yet to remove uploaded files from S3.
- This is also considered a call to S3, so avoid using it as much as possible.
3. **Add another table in database for metadata**
- Changing metadata in AWS is very expensive, so its better to ***only*** store file uploads in S3 and keep the metadata in a separate database.
- Folder directory can be improved by adding a path column for easier bread crumb navigation.
- `/components/dms/dialogs/CreateNewDialog.vue` currently displays the correct folder structure according to the database, but because metadata is not available, the navigation for creating a new folder does not work when clicked.
4. **Modifying UI to fit with data and metadata from database**
- A lot of the frontend UI components use hard-coded dummy data that currently does not match the current data available in the database. Consider adding another table for metadata.
- Below is a non-exhaustive list of personal recommendations:
```
Requesting additional changes to the database:
ADD item_count for counting total items in current folder.
ADD access_level to determine access level for each user role.
QUERY cb_sector is still unknown for what uses.
ADD children_count to determine the amount of children for the current folder.
REPLACE cb_parent_id or ADD path for each folder.
AWS S3 recommendations:
- Add AWS Cloudfront to reduce S3 calls and usage, saving costs.
- Add function to delete files in S3.
```
# Contact Me
Some considerations might have went over my head. If you find yourself in a pinch or with a question, feel free to contact me through my email:
- (shariffbukhary01@gmail.com)[mailto:shariffbukhary01@gmail.com?subject=CORRAD%20EDMS]
Thank you for your continued support and work on this project!
PS: There is a docs folder that also contains this markdown file and also a bunch of other stuff you might need. Have a look around.