Processing times for data ingestion can vary greatly.This KB article is provided to help you understand why some actions take time, and your options to help improve the process speed.
When Diligex imports data for any client, there are many factors that need to be taken into account, like the number of individual PST, NSF files and EML folders overall, size of a PST file, complexity of a folder structure housing the messages, number of documents and number of messages.
Timeframes for Import:
4+ weeks - under 25GB
6+ weeks - under 50GB
8+ weeks - under 100GB
12+ weeks - over 100GB
TBD for anything over a Terabyte
Contact us for an estimated completion time
Note: These are rough estimates and should not be considered a commitment or SLA.
Why do some imports go faster other than size differences?
Once files are converted into a standard MIME/RFC.822 format they are inserted into a queuing service for archiving and indexing. The processing rate varies at this state based upon a few factors. They include:
- Size/profile of email attachments that go through a text extraction service.
- Availability of excess compute services.
- Capacity consumed by day to day archiving activities (daily journaling, exports, etc.)
Can you help improve the efficiency of importing this data?
The most efficient method to process any email data is to provide it to Diligex in a standard MIME/RFC.822 format, typically .eml with all the original metadata intact. We recommend that single PST files not exceed 20GB, and have been proven to not corrupt when an outlook client attempts to open it. In addition, PST Files should NOT be password protected.
Can Diligex speed things up?
When the data is provided to Diligex in the requested and preferred formats described above, we are able to accelerate the data processing. If you provide data that has urgent e-discovery needs, we improve the importing of your data by our utilizing additional compute resources, although this incurs an extra premium cost.