Go spreadsheet and CSV libraries for messy business data

Why messy files cause trouble

Business files look simple until real people touch them. Sales exports one format, finance keeps another, and operations adds its own column names. When teams test Go spreadsheet and CSV libraries, this is usually the first problem they hit.

The same field often shows up under several labels. One file says "email," another says "work email," and a third says "primary contact." A simple parser expects one exact header, so it fails even when the data is there.

Values create even more trouble. Dates can mean different things depending on who made the file. Is 03/04/2024 March 4 or April 3? Currency breaks too. "$1,200.00," "1200," and "1 200,00" may all mean the same amount. Blank cells add another layer because a blank can mean "unknown," "not required," or "same as above."

A strict import looks neat in code, but people hate it in practice. If one bad row stops a 2,000 row upload, the user has to fix that problem before they learn what else is wrong. Onboarding turns into a slow loop of upload, fail, edit, repeat.

A better import checks each row on its own. If 1,997 rows are fine, save them and report the three bad ones in plain language:

• Row 18: "Start date must use YYYY-MM-DD"
• Row 44: "Amount is not a valid number"
• Row 107: "Missing company name"

That difference changes support load fast. A generic "import failed" message sends people to support. Clear row errors let them fix the file on their own in a few minutes.

This shows up all the time during customer onboarding. A startup might get a contact list from one tool, billing data from another, and a hand edited spreadsheet from the customer success team. The file is not clean, and it never will be. Good import work starts by accepting that.

What to decide before you pick a library

A bad library choice often starts with a bad assumption: "our customers will upload a simple CSV." They rarely do. Teams send sales exports, XLSX files, copied reports with odd headers, and spreadsheets full of empty rows. Start with the files people already send, not the files you wish they sent.

Build a small sample set before you compare packages. Include one clean file and a few ugly ones. One customer may upload 500 rows. Another may send 120,000 rows with duplicate emails, mixed date formats, and columns in the wrong order. That size changes what your app can hold in memory and how fast users get feedback.

XLSX is not just CSV with extra steps. Use CSV when the file is plain table data and speed matters more than layout. Use XLSX when customers rely on multiple sheets, fixed templates, saved cell types, or workbook files from finance and operations teams. If people already work in spreadsheets every day, forcing everything into CSV can create more support work.

Import and export usually need different tools. A package that handles large imports well may feel awkward when you need polished exports. A package that creates nice XLSX reports may use too much memory for onboarding uploads. Treat those as separate jobs.

Before you write any code, decide what users must see when a row fails. Most teams need more than "invalid file." They need the row number, the column name, the rejected value, a short reason, and an example of the accepted format. That decision shapes the rest of the system. It affects how you parse rows, when you validate data, and whether you stream the file or load more of it first.

A founder does not care which library you picked. They care that a 40 MB onboarding file imports without stalling, and that their team can fix row 218 instead of opening a support ticket.

CSV options in Go

Most teams should start Go CSV import work with the standard encoding/csv package. It is small, predictable, and easy to control when business files arrive with extra columns, odd headers, or rows that mix empty values with bad ones. You read row by row, validate fields yourself, and keep the row number for clear error messages.

That direct approach is often the right one when onboarding data is messy. You decide how to trim spaces, whether to allow missing columns, and what to do when header names do not match your app exactly. It sounds less convenient at first, but it usually saves rework later.

A simple rule works well. Use encoding/csv when you need full control. Use csvutil when you want to map rows into structs with tags and keep the code shorter. Use gocsv when you prefer a struct based flow and the files are fairly regular.

Libraries that map straight into structs can speed up clean imports. They work nicely when columns are stable and the file format rarely changes. They get awkward when users rename headers, add blank columns, or send exports from different systems. In those cases, many teams parse with encoding/csv first, normalize the headers, and then map the cleaned row into a struct.

Large files need streaming. Do not read the whole file into memory unless the files are tiny and you know they will stay tiny. Read one record at a time, validate it, store what passed, and collect row errors as you go. That keeps memory use flat even when a file has hundreds of thousands of rows.

Be explicit about the format. Set or detect the delimiter on purpose. Decide how quotes should work. Check whether the first row is a header, and reject duplicate or missing column names early. Many import bugs come from silent assumptions, not from the parser itself.

Clean sample files are not enough for tests. Add rows with broken quotes, shifted columns, empty required fields, duplicate headers, and strange delimiters. If your parser survives those, it will survive real customer uploads.

Spreadsheet options in Go

Excelize is the common choice when teams send real XLSX files instead of plain text exports. It handles the usual work well: opening a workbook, reading cells, writing templates, and generating finished files people can download without fuss.

Read only the sheets you actually need. Many business workbooks include extra tabs for notes, old exports, lookup tables, or somebody's scratch work. If your importer expects "Customers" and "Contacts," ignore everything else. That keeps memory use lower and cuts down on strange parsing errors from tabs nobody meant to upload.

Excel files also hide a few traps that do not show up in a quick manual check. Merged cells can make one visible value span several rows even though only one cell really contains data. Formula cells may hold a formula, a cached result, or both, depending on how the file was saved. Hidden rows still exist, so you need a rule for them. Most teams should either skip hidden rows every time or import them every time. Do not switch behavior from file to file.

A short set of rules helps: read named sheets only, treat the first row as the header row, reject merged cells in data areas, accept formula results only when they resolve to plain values, and skip hidden rows only if you document that rule.

Excelize is useful on the export side too. Generate a template that matches your import headers exactly, in the same order, with one example row if needed. That sounds minor, but it saves a lot of back and forth. People usually copy an old file when they do not have a template, and old files carry old mistakes.

If workbook features add no real value, use CSV instead. A one sheet import with fixed columns is often easier to support as CSV. You avoid formatting surprises, reduce parsing overhead, and make validation simpler for both your code and your users.

Keep memory use under control

Most memory problems start with one habit: loading the whole upload, turning every row into a struct, and only then checking whether the file is usable. That works in tests. It breaks when onboarding brings a 40 MB CSV with extra columns, empty cells, and repeated headers.

Safer code reads rows one at a time. For CSV, that means using a reader that returns one record per call. For XLSX, use row iterators or stream modes when the library offers them. You only need the current row, the row number, and a small error buffer.

Set limits before parsing starts. If a file is too large, has too many rows, or has suspiciously wide records, stop early and return a clear message. Cap file size, row count, column count, and cell length. Reject empty files and duplicate header rows. Stop after a sensible error threshold. These checks save memory and time, and they protect the rest of your import flow from junk files.

Another common mistake is creating full app structs too early. Parse the raw row first. Validate the fields you need. Save only the rows that pass. If a row fails, keep a small error record such as row 182, column "email," invalid format. That uses far less memory than storing every parsed object plus every validation result.

Split the work into three steps: parse, validate, save. Each step should handle a small amount of data. Save good rows in batches instead of keeping thousands in memory while you wait for the last row. If you need deduplication, store compact lookup values such as emails or external IDs, not full rows.

Measure with real onboarding files, not toy samples. A clean 200 row export tells you almost nothing. Test with the files teams actually upload: wide finance sheets, messy CRM exports, and CSVs with broken quotes. Watch peak memory, processing time, and garbage collection spikes. That is where Go file parsing memory use becomes a real product issue.

Build the import flow step by step

A good import flow keeps bad data out of your database and cuts support work. Most Go spreadsheet and CSV libraries can open a file, but the steps around that read decide whether onboarding feels smooth or painful.

Start by treating every upload as its own record. Save the raw file first and assign an upload ID before you parse anything. That gives you a clear audit trail, lets support reproduce a problem, and makes retries much easier when a user uploads a corrected file.

Then move through the file in a fixed order. Read the header row and map it to the fields your app expects. Users rarely name columns the same way every time, so map common variants like "Email," "Work Email," or "E-mail" to one field. Parse each row into a small internal shape with fields like NameRaw, EmailRaw, StartDateRaw, and DepartmentRaw instead of full database models. Validate one row at a time. Check required fields, bad dates, broken email formats, and duplicates inside the same upload. Keep the row number with every error. "Invalid email" is not enough. "Row 18: invalid email" gives people something they can fix quickly. Before you write anything to the database, show a preview such as "142 rows ready, 11 rows need fixes."

That preview does real work. If a customer sees that "start date" got mapped to "department," they can stop right there instead of creating cleanup work for your team.

Only write accepted rows after the user confirms the preview. For small imports, one transaction often works. For larger files, write in batches and record which rows succeeded under the same upload ID.

Give users an error file they can correct and upload again. Keep the original columns, add an "Error" column, and preserve the original row order. If row 27 has a missing email and row 31 has a bad date, the person fixing the sheet should see both problems in the same file without guessing what your system rejected.

That is what makes Go CSV import feel reliable during onboarding: clear mapping, small parsing steps, and row errors people can act on.

Return row errors people can fix

When an import fails, people do not want a technical log. They want to know what broke, where it broke, and what to change in the file. If a team uploads a 2,000 row sheet and gets back "invalid input," someone usually ends up fixing it by hand.

A useful error points to the exact row and column. "Row 18, column start_date: date missing" is much better than "validation failed." If the value exists but has the wrong shape, show that too: "Row 18, column start_date: got 13/32/2024, expected YYYY-MM-DD."

That small bit of context saves a lot of back and forth. People can open the file, jump to the row, and correct it in a minute.

What good row errors look like

A clear import report should include the row number from the uploaded file, the header name people see, the bad value that caused the error, the expected format or allowed values, and a short message in plain language.

Plain language matters here. "Price is not a number" beats "type coercion failed." "Date missing" beats "required field validation error."

You should also separate accepted rows from rejected rows. If 1,940 rows are fine and 60 have problems, keep the 1,940 ready for import and put the 60 in a reject list or a clean error report. Do not force users to rework the whole file because a few rows are bad.

Repeated errors need two views at once: a summary and row detail. A summary helps people spot patterns fast, such as "43 rows have an invalid country code." Row detail still matters because they must fix row 12, row 47, and row 301 one by one.

A simple layout works well: accepted rows ready to import, rejected rows with errors only, and an error summary with counts by error type.

For onboarding data, this can cut support time fast. If a sales ops manager uploads a customer list and sees "Row 24, column price: got free, expected number," they can fix it without asking engineering.

A realistic onboarding example

A B2B SaaS team is onboarding 4,000 customer contacts from three departments. Sales exports a CSV with a "Company" column. Support sends an XLSX file that calls the same field "Client," while finance uses "Account."

People mean the same thing, but the headers do not match. That is where messy imports usually start.

In Go, the import service begins with a header map. It treats "Company," "Client," and "Account" as one internal field, customer_name. That small step can save hours of manual cleanup before anyone uploads a file.

Each row then goes through a few checks before the app creates anything. The service trims spaces, normalizes email case, and checks whether the email already exists. If two rows share the same email, or the email is already in the database, the import rejects that row before account creation.

The batch still keeps moving. One bad row should not stop 3,999 good ones.

When the run ends, the sender gets two results. First, they see a plain summary: how many rows passed on the first try, how many need fixes, and how many were blocked as duplicates. Second, they get an export of the same file with an added "Error" column next to the original data.

That export matters because people can actually use it. A sales manager can sort the sheet, scan messages like "duplicate email" or "missing company name," fix the cells, and upload again without asking engineering for help.

The first pass number tells the team a lot. If 3,200 of 4,000 rows pass, the mapping rules are probably in good shape. If only 900 pass, the import flow needs work. Maybe one department changed its export format, or maybe the required fields are too strict for real business data.

That is a practical onboarding flow. It accepts that files will be inconsistent, blocks bad account creation early, and gives people a corrected file they can use.

Mistakes that create rework

Many teams compare Go spreadsheet and CSV libraries as if both file types follow the same rules. They do not. A CSV is usually one flat table, but an XLSX file can hide merged cells, empty header rows, formula cells, and extra sheets people forgot to remove.

That difference causes quiet damage. A file may look fine in Excel, then break your importer because the real header starts on row 4 or because the user uploaded the wrong sheet.

Another common mistake is writing rows to the database first and validating later. That saves a little time at the start, but it creates cleanup work every time the file has bad emails, missing account IDs, or dates in the wrong format.

A safer order is simple: parse the file, normalize values, validate each row, write only the clean rows, and return errors for the rest. If you skip that order, support teams end up fixing partial imports by hand.

Returning one error for the whole file is just as costly. People cannot fix what they cannot find.

Quick checks and next steps

Most teams should pick one default import path. For messy business data, CSV usually wins for intake because it is easier to inspect, easier to repair, and lighter on memory than full spreadsheet handling. If you still support XLSX, treat it as a secondary option, not the main route.

A short review now can save a week of cleanup later. With Go spreadsheet and CSV libraries, the library choice is only half the job. The bigger issue is whether your import flow matches the files real customers already have.

A few checks go a long way. Pick one default format for imports and document it in plain language. Test with a real customer file, including bad headers, blank rows, strange dates, and duplicate records. Check memory limits before the first large onboarding batch, not after the process crashes. Make your exports use the same field names, order, and rules as the import template. Return row errors that tell people exactly what to fix.

Teams often miss the export point. If your app accepts "customer_name" on import but exports "client" later, users will edit the wrong file and upload a broken version next time. A matching round trip is boring, but it works.

Memory checks should be blunt. Load a file close to the biggest one you expect, run the import in the same environment you plan to use in production, and watch RAM use. If the process gets close to the limit, switch to streaming earlier. Do not wait for the largest customer to find that bug for you.

One real file tells you more than ten clean samples. A customer spreadsheet usually includes extra tabs, hidden spaces, mixed phone formats, and columns nobody warned you about. That is the test case that counts.

If you are planning this kind of import work, it helps to have somebody who has already dealt with messy production systems at scale. Oleg Sotnikov at oleg.is works with startups and small teams on architecture, infrastructure, and practical AI driven development, which often includes exactly this kind of onboarding and data pipeline cleanup.