Discover how the CleanMyCSV engine analyzes, fixes, and secures your databases. This guide breaks down the full processing pipeline applied to your files, guaranteeing usable, reliable, and standardized data.
1. Fixing Encoding & Character Issues
Legacy system exports or old Excel versions (Windows-1252/MacRoman) often spit out encoding errors, turning accents into unreadable gibberish.
How we fix it:
CleanMyCSV scans your file to detect encoding anomalies (e.g., weird `` characters). If an outdated format is detected, it force-converts everything to clean UTF-8, ensuring all special characters and international accents are perfectly preserved.
Poorly formatted CSVs often dump all data into a single column. This happens when the target software expects a different separator than the source file.
How we fix it: The algorithm parses a data sample (up to 10,000 characters) and calculates a probability score to pinpoint the exact delimiter. It seamlessly handles commas embedded inside text fields (like addresses) without breaking your columns.
3. Typographical Normalization (Names & Zip Codes)
Visual and technical consistency in text fields is non-negotiable for a healthy customer database.
Proper Case (Names): The engine harmonizes name capitalization. It smartly handles complex particles (De, Du, Des, Le, La, Van, Von) and Anglo-Saxon prefixes (Mc, O').
Example: mcdonald O'neil becomes McDonald O'Neil.
Zip Code Restoration: Spreadsheets like Excel love to treat zip codes as numbers, stripping away leading zeros (a huge headache for international or US East Coast zip codes). CleanMyCSV catches this and restores the missing zero to match legal lengths.
Example: 7500 becomes 07500.
4. International Phone Normalization
The engine automatically spots phone-related columns (Mobile, WhatsApp, etc.) using strict regex and applies smart formatting.
Country codes: Replaces 00 and (0) with the standard + format.
Duplicate handling: If a cell contains Num1 / Num2, only the first one is kept for a clean CRM import.
ID Protection: Alphanumeric strings over 13 characters without separators are ignored to protect your business IDs (SSN, EIN, etc.).
Price columns packed with currency symbols or spaces are read as "text" by analytics tools, breaking all your math.
How we fix it:
1. Detects currency symbols ($, €, £, ¥) or ISO codes (EUR, USD, GBP, JPY, CAD, AUD, CHF, SEK).
2. Extracts the currency to isolate it. (Note: depending on the file layout, this may create a brand new "Currency" column).
3. Strips the amount of any spaces or rogue thousand separators, standardizing the decimal mark into a clean, math-ready format (dot).
Before
1 250,50 € | $ 45.00
After (Math-ready)
Amount;Currency
1250.50;EUR
45.00;USD
6. International Date Standardization (ISO 8601)
Date format conflicts destroy sorting and automations.
How we fix it: The script sniffs out various date formats—including mixed formats, text dates ("Jan 1, 2024"), and Excel serial numbers (e.g., 45292). It then converts everything into the universal database standard: YYYY-MM-DD.
7. Regional Adaptation: FR vs. US Logic
Data formatting standards differ wildly between Europe and North America. A generic cleaning tool will corrupt your financial amounts or misread your file structure.
How we fix it: CleanMyCSV runs on a dual-engine setup. Before processing, you define your data context (FR or US Logic).
Impact on your data:
Numbers: In FR mode, the engine knows spaces separate thousands and commas mark decimals (1 250,50 becomes 1250.50). In US mode, it correctly reads the comma as a thousand separator (1,250.50 becomes 1250.50).
Separators: The algorithm tweaks its detection weights, favoring semicolons (;) for FR files and commas (,) for US files.
Bloated files slow down your systems. We clean up the dead weight on the fly.
100% blank rows: Any row without meaningful data is ignored and dropped from the output.
Strict deduplication: The engine runs a full-row comparison. If two or more rows are identical across all columns, only one survives.
9. Data Security & XSS Prevention
Importing external files is a massive security risk (script injections).
How we fix it: Before outputting, we apply surgical sanitization. Sensitive coding characters (<, >, &, ", ', /) are converted into harmless HTML entities. This guarantees your CSV can't execute malicious code when imported into your CRM or displayed on a web dashboard.
10. Audit Trail (The JSON Report)
Total transparency. You aren't feeding your data into a black box.
How we fix it: Every tweak CleanMyCSV makes is logged. Alongside your clean file, the tool generates an exhaustive audit report (ready for download).
What's inside: The exact row count, column count, and a line-by-line breakdown of the 'Before' and 'After' values, complete with the exact reason for the change (capped at 1,000 details to keep things lightning fast).