MIME & Rest API
Building a Stream-Based File Upload Integration using IBM webMethods MFT and MIME Multipart REST APIs
2026-07-29 · 10 min read
Introduction
Enterprise integrations are no longer limited to exchanging JSON or XML payloads. Many modern SaaS platforms expose REST APIs while still expecting documents to be uploaded as physical files using multipart/form-data requests.
In one of my enterprise integration projects, the HR system periodically generated a CSV file containing employee badge information such as Employee ID and PIN. Instead of consuming individual employee records, the downstream Workforce Management platform expected the complete CSV file to be uploaded through a REST API as a multipart MIME attachment.
This presented an interesting integration challenge.
How do you upload a potentially large file through a single HTTP request without loading the complete file into JVM memory?
IBM webMethods provides an elegant solution using Managed File Transfer (MFT), InputStreams and MIME services.
This article explains the architecture, runtime processing and the concepts behind stream-based file uploads.
Business Problem
The HR system exports employee badge information every few hours in CSV format.
Example:
EmployeeId,Pin
100001,1234
100002,5678
100003,9876
The receiving Workforce Management platform does not expose a file share or FTP server.
Instead, it exposes a REST endpoint which expects the CSV to be uploaded using multipart/form-data.
The integration therefore had to:
- Automatically detect new files.
- Upload the file without manual intervention.
- Preserve the original CSV.
- Avoid converting the file into JSON.
- Handle large files efficiently.
- Minimize JVM memory consumption.
High-Level Architecture
HR System
│
Generates Employee CSV
│
▼
Managed File Transfer
(Scheduled Event)
│
Reads File Stream
│
▼
Integration Server
│
createMimeData()
│
addBodyPart(fileStream)
│
getEnvelopeStream()
│
▼
HTTP Multipart POST
│
▼
Workforce Management Platform
Complete Integration Flow
The complete integration follows a simple but highly optimized sequence.
HR System
│
│ Generates Employee Badge CSV
▼
Managed File Transfer (MFT)
│
│ Scheduled Event
▼
InputStream
│
▼
pub.mime:createMimeData()
│
▼
pub.mime:addBodyPart()
│
▼
pub.mime:getEnvelopeStream()
│
▼
HTTP POST
│
▼
Workforce Management Platform
Each stage is responsible for a single concern:
- MFT reliably detects and retrieves files.
- Integration Server processes the file without loading it into memory.
- MIME services prepare the multipart request.
- The HTTP client streams the file directly to the destination API.
Step 1 – Creating the MIME Container
The first MIME service invoked is:
pub.mime:createMimeData
This service creates an empty MIME container.
Think of it as creating an empty envelope before inserting a document.
MimeData
↓
Empty Multipart Container
At this point no file has been attached.
Step 2 – Attaching the File Stream
The next service invokes:
pub.mime:addBodyPart()
This service attaches the InputStream as a MIME body part.
A common misconception is that the service copies the complete file into memory.
It does not.
Instead, it stores only a reference to the InputStream.
MimeData
↓
Body Part
↓
InputStream
The actual file is read only when the HTTP request is transmitted.
Step 3 – Building the Multipart Request
Finally,
pub.mime:getEnvelopeStream()
returns another InputStream called the Envelope Stream.
This stream represents the complete multipart HTTP request.
Conceptually it contains:
Boundary
↓
Headers
↓
CSV File Stream
↓
Closing Boundary
The multipart request is generated dynamically while being transmitted.
No large byte array is created in memory.
Why Managed File Transfer (MFT)?
IBM webMethods Managed File Transfer provides a reliable mechanism for receiving files from enterprise applications.
Instead of writing custom polling logic, MFT continuously monitors configured directories or SFTP locations and automatically triggers Integration Server services whenever a new file arrives.
This provides:
- Reliable file detection
- Automatic retries
- Event-based processing
- Audit capabilities
- Enterprise-grade monitoring
Why InputStream Instead of String?
One of the biggest mistakes developers make is reading the complete file into a String.
For small files this is acceptable.
For large files it becomes expensive.
Traditional
CSV
↓
String
↓
Heap
↓
REST
Versus
CSV
↓
InputStream
↓
HTTP
↓
Socket
Understanding MIME Multipart Requests
HTTP file uploads are typically implemented using the multipart/form-data specification.
Instead of sending raw bytes directly, the request body contains one or more MIME parts.
Each part contains its own metadata such as:
- Content-Disposition
- File Name
- Content-Type
followed by the actual file bytes.
IBM webMethods automatically generates this structure through its MIME services.
Multipart
↓
Header
↓
Boundary
↓
CSV Stream
↓
Boundary
Runtime Processing Inside Integration Server
Although the integration performs a single HTTP POST request, the complete file is never loaded into memory.
The HTTP client continuously reads small chunks from the InputStream and writes them to the network socket.
The connection remains open until End-of-Stream (EOF) is reached.
This means a multi-gigabyte file can be transmitted while memory usage remains almost constant.
Read
↓
8 KB
↓
Socket
↓
Read
↓
8 KB
↓
Socket
Runtime Processing
Although the Flow invokes a single REST service, the entire file is never loaded into JVM memory.
Internally the HTTP client repeatedly performs an operation conceptually similar to:
while(true){
bytes = envelopeStream.read(buffer);
if(bytes == EOF)
break;
socket.write(buffer);
}
Only a small buffer (typically 8 KB–32 KB) exists in memory at any given time.
This allows multi-gigabyte files to be uploaded with nearly constant heap utilization.
Benefits of This Design
- Constant JVM memory usage
- Suitable for large files
- Standard multipart/form-data implementation
- Reliable automated file detection
- Production-ready architecture
- Easily reusable for PDF, ZIP, XML, Excel and CSV files
- Minimal Garbage Collection overhead
Failure Handling
MFT
↓
Download
↓
Failure
↓
Retry
↓
Archive Failed
↓
Mail
Final Architecture
HR System
│
Generates CSV File
│
▼
Managed File Transfer (MFT)
Scheduled Event Trigger
│
▼
Reads File as InputStream
│
▼
Integration Server Flow Service
│
┌──────────────────────────┐
│ createMimeData() │
│ addBodyPart() │
│ getEnvelopeStream() │
└──────────────────────────┘
│
▼
Multipart HTTP POST Request
│
▼
Workforce Management Platform
│
▼
Employee Badge Updated
Conclusion
This integration demonstrates how IBM webMethods can efficiently bridge traditional file-based systems with modern REST APIs without compromising performance or memory utilization.
By combining Managed File Transfer, InputStreams and MIME multipart services, the platform streams the file directly to the target application, making the solution scalable, reliable and suitable for enterprise workloads.
This design pattern is reusable across HR, ERP, Banking, Manufacturing and Retail integrations where files must be transferred through REST APIs while maintaining low memory consumption and high throughput.