Overview
This project focused on building a fast and stable upload infrastructure for a social media platform where users can send large video and media files. The core challenge was keeping the experience smooth even when file sizes reached 2GB or more.
The frontend was developed in React, while the backend used CodeIgniter. The initial goal was to support upload, compression, and storage in a way that would not overload the backend server or interrupt the user experience.
After testing multiple approaches, the final architecture shifted heavy upload work away from the backend by using AWS Presigned URLs for direct upload, with FFmpeg compression handled asynchronously through a separate cron process.
Core Objectives
- Handle large media uploads with minimal backend pressure
- Improve upload stability for long-running file transfers
- Compress videos without blocking the main upload flow
- Keep the architecture scalable for social media traffic
Features
A resilient media pipeline.
Direct AWS Presigned Uploads
The frontend requests a presigned URL from the backend and uploads the file directly to AWS. This removes the need for the backend to act as a file relay and immediately improves speed and scalability.
Chunk-Based Upload Experimentation
A second approach split large videos into smaller chunks for safer transfer. This improved reliability compared with backend-only upload, but still introduced complexity for merge and processing logic.
FFmpeg Compression Workflow
Video optimization is handled using FFmpeg so that uploaded media can be compressed efficiently without slowing down the interactive upload flow.
Cron-Driven Processing
A separate cron job checks for newly uploaded uncompressed videos, downloads them, compresses them, and pushes the optimized version back to AWS.
Stable Media Storage Layer
The system stores file metadata after upload completion so the application can reliably track media assets across the platform lifecycle.
Challenges
Slow Large File Uploads01
Uploading very large videos through the backend made the process slow and fragile.
Moved upload delivery directly to AWS using presigned URLs so the backend no longer became the bottleneck.
Backend Load Spikes02
Handling uploads, compression, and file storage together placed too much stress on the application server.
Separated storage and compression responsibilities so the backend only orchestrates the workflow.
Connection Interruptions03
A failed connection, page refresh, or browser close could interrupt the full upload flow.
Improved transfer stability by using direct-to-storage uploads and chunked upload experimentation.
Complex Compression Flow04
Trying to upload, compress, and persist files in one step created long processing times.
Moved compression into a separate cron-based FFmpeg workflow after upload completion.
Strategy
Presigned URLs, async compression, and cleaner scaling
The final architecture prioritized direct file delivery to AWS, deferred video optimization, and a lighter backend footprint. That balance made the upload flow faster for users and easier to scale for the platform.
Backend De-bottlenecking
By moving the actual file transfer to AWS, the backend no longer had to receive and stream massive media payloads.
User Experience Protection
Uploads became less fragile because the direct-storage approach reduced the number of failure points during transfer.
Operational Simplicity
The backend stayed focused on orchestration, metadata, and workflow control instead of heavy media handling.
Scalable Media Architecture
The final solution supports growth in file volume and size while leaving room for future optimization layers.
What I Learned
- A solution that works is not always the most efficient one.
- Presigned URLs are a strong pattern for direct, scalable uploads.
- Compression is often best handled asynchronously after upload completion.
- Performance optimization matters a lot in media-heavy social platforms.
Conclusion
The final social media upload system delivers a faster and more stable way to handle large video files.
By combining AWS Presigned URLs, FFmpeg compression, and cron-driven processing, the platform now supports large media uploads with less backend strain and a better user experience.