Summary
Large files (any file greater than 5 million rows) often present time-consuming challenges.
Learn some tips and tricks for how to get large files into MindBridge with ease.
Tips
- Zip or compress your data file. MindBridge has implemented an 8GB file import limit to reduce the likelihood of import failures due to network connectivity issues. You can zip your file, effectively reducing the file size by a factor of 10-15x, and import the compressed file into MindBridge.
- If the general ledger is spread across multiple files that need to be combined (for example, one file per month), you will need to zip each file individually before loading them into the File manager; if no formatting needs to be done, use the merge tool to combine them. You can also use the split tool for workbooks with multiple worksheets.
- If possible, use a wired connection when importing large files. Wired networks are faster, more stable and reliable, and offer better security than wireless networks.
- Before obtaining an export, identify which columns you need for your analysis, including the columns required in MindBridge. Large files can take a lot of time to import, and even longer if any data formatting is needed. Keeping only the columns you absolutely need will save time and disk space.
- Identify a good transaction ID. Although it is always important to identify a good transaction ID, the larger the file, the more important this becomes to successfully running an analysis. Read the following articles for more information:
- The first Data Validation step provides high-level information to help you perform quick sanity checks and to spot potential data ingestion and integrity issues before completing the import. Since the full import process can take a few hours for large files, spotting potential issues before proceeding with the import can save you a significant amount of time.
- Plan ahead and be patient. If there are no data issues, you might spend a day completing the import process, and then the analysis might need to run overnight or longer.
- If your GL is exceptionally large (approximately 50 million rows or more), feel free to let us know before running your analysis so we can keep an eye on it and/or let you know if there are other large jobs already running that could cause delays.
- Check with your IT team about security software that may be impacting the import process. For example, you will not be able to import a 5GB file into MindBridge if your browser limit is set to 4GB.
Anything else on your mind? Chat with us or submit a request for further assistance.