Exporting a few hundred records to CSV is straightforward. Exporting hundreds
of thousands or millions is different: if you build a
List<T>, a
StringBuilder, or a
MemoryStream for the complete result,
application memory grows with the export and the server may have to finish
generating the file before the download can begin.
This tutorial shows how to stream large CSV exports in ASP.NET Core with Entity Framework Core and SQL Server. You'll build a .NET 10 minimal API that reads an EF Core query asynchronously and writes rows directly to the HTTP response. You'll also add request cancellation, CSV quoting and formula-injection mitigation, a per-process concurrency limit, and an optional keyset-batched mode for cases where keeping one database reader open for the whole download is undesirable.
AsAsyncEnumerable(),
and write each CSV record to the response with cancellation-aware
StreamWriter methods. Don't call
ToListAsync() for the complete result. EF
Core and the hosting stack can still buffer internally in some configurations.
Fetch and format the first row before creating the BOM-enabled writer so an
initial query failure can still be handled before CSV bytes are sent. Once the
response has started, a later failure can leave the client with a partial file
rather than a replacement HTTP error response.
Prerequisites
- The .NET 10 SDK and a SQL Server instance where you can create a development database (LocalDB on Windows, or SQL Server in a container or on a server).
- Basic C#, HTTP endpoints, and EF Core migrations. You don't need to know streaming already.
-
A terminal and a tool that can download a URL, such as
curl.
What you will build
The sample exposes GET /api/orders/export for
a fictional Codingvila store. It accepts optional date boundaries, defaults to
the previous 30 days, rejects reversed ranges and ranges longer than 366 days,
and returns a downloadable UTF-8 CSV file. It also demonstrates:
- EF Core projection and asynchronous enumeration.
- CSV quoting, CRLF line endings, invariant number and date formatting, and a UTF-8 byte order mark (BOM) for spreadsheet compatibility.
- Defensive handling of text values that spreadsheets could treat as formulas.
- Cancellation-aware reads and writes.
- A per-process concurrency limiter and an authorization policy that's open only in Development.
- An optional keyset-batched query that keeps each database reader open for a shorter time.
The diagram below shows the request path. The request travels down to SQL Server, and rows travel back up as they are read.
1. When should you stream large CSV exports in ASP.NET Core?
Streaming helps when the result set is large or unpredictable and you don't
want to hold the full result and the full CSV in application memory. A
buffered export usually calls ToListAsync(),
builds a byte array or memory stream, and only then starts sending the
response.
| Concern | Buffered export | Streamed response |
|---|---|---|
| Application memory | Grows with the full result and output buffer | Can remain bounded by query, writer, and transport buffers |
| First response bytes | Usually sent after file generation | Can be sent while later rows are still being read |
| Client disconnect | Must be wired through generation and database work | Can propagate through the request cancellation token |
| Retry or resume | Can be implemented with a completed temporary file | A failed in-progress response is generally not resumable |
| Database connection | Depends on when the query is executed and materialized | A single-reader export usually holds a connection while the client downloads |
Streaming doesn't mean nothing is buffered. The
StreamWriter has a buffer, EF Core and the
database provider may buffer in some configurations, and your server or
reverse proxy may buffer too. The goal is narrower: never materialize the
entire result set and the entire CSV in application memory.
For a small, strictly bounded collection, a plain buffered export is simpler. Codingvila's older walkthrough on exporting data to a CSV file with ASP.NET MVC shows that buffered style: it builds the whole file as a string and returns it as bytes. This article is about database-backed exports that can keep growing.
2. Create the .NET 10 project
The sample targets .NET 10 LTS, EF Core 10, and SQL Server. EF Core 10
requires the .NET 10 SDK to build and the .NET 10 runtime to run; see the
EF Core 10 release notes. Check the
official .NET support policy
for the current servicing release before you deploy. The commands below use
floating 10.0.* versions so NuGet resolves
the latest stable 10.0 patch. For a reproducible build, commit the exact
package and tool versions selected for your project rather than relying on a
floating version in long-lived automation. If you're moving an existing
application to this target, start with Microsoft's
breaking changes in .NET 10
list.
Create a minimal API project, then add the SQL Server provider, the EF Core
design-time package, the JWT bearer package (used by the non-Development
configuration in section 6), and the
dotnet-ef tool:
dotnet new web --framework net10.0 -n Codingvila.CsvExport
cd Codingvila.CsvExport
dotnet add package Microsoft.EntityFrameworkCore.SqlServer --version "10.0.*"
dotnet add package Microsoft.EntityFrameworkCore.Design --version "10.0.*"
dotnet add package Microsoft.AspNetCore.Authentication.JwtBearer --version "10.0.*"
dotnet tool install --global dotnet-ef --version "10.0.*"
Keep the EF Core provider, the design package, and
dotnet-ef on compatible 10.x versions. The
sample uses C# primary constructors and
SearchValues<char>, both supported by
the .NET 10 toolchain.
For local development on Windows, add a LocalDB connection string to
appsettings.json:
{
"ConnectionStrings": {
"Default": "Server=(localdb)\\MSSQLLocalDB;Database=CodingvilaCsvExport;Trusted_Connection=True;TrustServerCertificate=True"
}
}
TrustServerCertificate=True is only there to
keep the local connection simple. Don't copy it into a production connection
string unless you understand what it turns off. On macOS or Linux, use a
reachable SQL Server instance and keep credentials out of source control with
.NET user secrets (or a managed secret store in production):
dotnet user-secrets init
dotnet user-secrets set "ConnectionStrings:Default" "Server=localhost,1433;Database=CodingvilaCsvExport;User Id=sa;Password=<your-password>;TrustServerCertificate=True"
The project will end up with this structure. EF Core creates the
Migrations folder when you add the first
migration.
Codingvila.CsvExport/
├── Codingvila.CsvExport.csproj
├── Program.cs
├── appsettings.json
├── Data/
│ └── AppDbContext.cs
├── Models/
│ └── Order.cs
├── Services/
│ └── OrderCsvExporter.cs
├── Csv/
│ └── CsvField.cs
└── Migrations/ (generated by dotnet ef)
3. Create the order model and DbContext
The example exports six columns. A narrow projection avoids loading unrelated data and makes the CSV schema explicit.
// Models/Order.cs
using System.ComponentModel.DataAnnotations;
namespace Codingvila.CsvExport.Models;
public class Order
{
public int Id { get; set; }
[MaxLength(200)]
public string CustomerName { get; set; } = string.Empty;
[MaxLength(320)]
public string? Email { get; set; }
public decimal Total { get; set; }
[MaxLength(30)]
public string Status { get; set; } = "New";
// Application convention: persist this value as UTC.
public DateTime CreatedUtc { get; set; }
}
Now configure the EF Core context and an index on the date column:
// Data/AppDbContext.cs
using Codingvila.CsvExport.Models;
using Microsoft.EntityFrameworkCore;
namespace Codingvila.CsvExport.Data;
public class AppDbContext(DbContextOptions<AppDbContext> options)
: DbContext(options)
{
public DbSet<Order> Orders => Set<Order>();
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<Order>(entity =>
{
entity.Property(o => o.Total).HasPrecision(18, 2);
entity.HasIndex(o => o.CreatedUtc);
});
}
}
The CreatedUtc index is a starting point, not
a promise that SQL Server will seek on it. The main query also orders by
Id, so depending on selectivity and table
size SQL Server may seek on the date index and then sort, or scan the primary
key in order and filter by date. Check the actual execution plan with
representative data before you add or change indexes. Microsoft's
efficient querying guide for EF Core
explains projection, tracking, and query-shape trade-offs in more detail.
SQL Server's datetime2 type stores no
time-zone offset. The sample assumes every
CreatedUtc value was written as UTC. If your
data holds local times or a mix, normalize it before you emit a literal
Z suffix. The formatter keeps all seven
fractional-second digits.
4. Format CSV fields safely
A CSV field needs quoting when it contains a comma, a double quote, a carriage return, or a line feed. A double quote inside a quoted field is written as two double quotes. The writer uses CRLF record separators and the common quoting conventions described in RFC 4180. RFC 4180 is an Informational RFC, so treat it as a widely used format reference rather than a rule that every CSV consumer implements identically.
There's also a security problem. If a user-controlled value starts with
=, +,
-, @, a tab,
or a carriage return (or a full-width variant of the first four), a
spreadsheet may evaluate it as a formula when someone opens the file. That's
CSV (formula) injection, and quoting alone doesn't stop it. See
OWASP's CSV Injection guidance.
This sample prefixes formula-like text with an apostrophe and quotes the
resulting field. OWASP documents this as a mitigation, but warns that Excel
may remove protective characters when a CSV is saved and reopened. For files
intended specifically for people to open in Microsoft Excel, OWASP describes
prefixing cells that begin with =,
+, -, or
@ with a tab inside the quoted field as an
Excel-resistant option. That tab remains part of the underlying data and can
affect later imports. The apostrophe may also remain visible in some
spreadsheet workflows. There is no universal sanitizer for every spreadsheet
application and downstream consumer, so choose and test the policy for your
actual export destination, including a save-and-reopen cycle. To change the
sample prefix, edit the FormulaGuard constant
below. The helper checks the first character of each text value; it does not
trim or normalize input. It is written for comma-delimited CSV, so add any
alternate delimiter to NeedsQuoting if you
change the file format.
// Csv/CsvField.cs
using System.Buffers;
namespace Codingvila.CsvExport.Csv;
public static class CsvField
{
// A compatibility mitigation for common spreadsheet workflows,
// not a universal guarantee for every spreadsheet application.
private const string FormulaGuard = "'";
private static readonly SearchValues<char> NeedsQuoting =
SearchValues.Create(",\"\r\n");
/// <summary>
/// Escapes a text field for CSV and prefixes formula-triggering text.
/// </summary>
public static string Escape(string? text)
{
if (string.IsNullOrEmpty(text))
return string.Empty;
var mustQuote = IsFormulaStart(text[0]);
if (mustQuote)
text = FormulaGuard + text;
if (!mustQuote && text.AsSpan().IndexOfAny(NeedsQuoting) < 0)
return text;
return string.Concat(
"\"",
text.Replace("\"", "\"\""),
"\"");
}
private static bool IsFormulaStart(char c) =>
c is '=' or '+' or '-' or '@' or '\t' or '\r' or '\n'
or '=' or '+' or '-' or '@';
}
Apply CsvField.Escape to every untrusted text
column. The exporter formats ids, decimals, and timestamps separately, so a
negative number stays a number instead of getting an apostrophe. A text value
that legitimately starts with a hyphen, such as
-Express, will also receive the prefix;
that's the cost of this simple first-character rule. A null email and an empty
email both become an empty field; if a consumer must tell them apart, define
an explicit representation in your CSV contract.
For more complex dialects, custom converters, or imports, a library such as CsvHelper is a sensible choice. CsvHelper offers asynchronous writing APIs; if you write straight to the HTTP response with it, use them, pass cancellation through, and never write synchronously to the response stream. Test the version you install, and apply the same formula-injection policy to untrusted text.
5. Build the streaming exporter
The exporter has two read modes. The default enumerates one database query as an asynchronous stream. The optional batched mode runs a keyset-paginated query for each batch of 2,000 rows. Both modes share the same projection, CSV formatter, writer, and cancellation token.
The exporter fetches and formats one row before it creates the
StreamWriter. This matters because the writer
uses a UTF-8 byte order mark (BOM): disposing a newly created writer can flush
its preamble even when no CSV row has been written. Creating it only after the
first query and formatting step succeed keeps an initial database or
formatting error from sending CSV bytes first. The exporter then writes the
header and first row and flushes them together. If the query returns no rows,
the download contains the header only. After the first flush, it flushes every
500 data rows. A flush doesn't guarantee a network packet:
StreamWriter, Kestrel, the operating system,
and any proxy may buffer data.
// Services/OrderCsvExporter.cs
using System.Globalization;
using System.Runtime.CompilerServices;
using System.Text;
using Codingvila.CsvExport.Csv;
using Codingvila.CsvExport.Data;
using Codingvila.CsvExport.Models;
using Microsoft.EntityFrameworkCore;
namespace Codingvila.CsvExport.Services;
public sealed class OrderCsvExporter(
AppDbContext db,
ILogger<OrderCsvExporter> logger)
{
private const string Header =
"Id,CustomerName,Email,Total,Status,CreatedUtc";
private const int FlushEveryRows = 500;
private const int BatchSize = 2_000;
private static readonly CultureInfo Invariant = CultureInfo.InvariantCulture;
private sealed record OrderRow(
int Id,
string CustomerName,
string? Email,
decimal Total,
string Status,
DateTime CreatedUtc);
public async Task WriteAsync(
Stream output,
DateTime fromUtc,
DateTime toUtc,
bool useBatches,
CancellationToken ct)
{
// Set a deliberate timeout for this export query, not globally for the app.
db.Database.SetCommandTimeout(TimeSpan.FromMinutes(5));
IAsyncEnumerable<OrderRow> rows = useBatches
? ReadInBatches(fromUtc, toUtc, ct)
: ToRows(Filter(fromUtc, toUtc).OrderBy(o => o.Id))
.AsAsyncEnumerable();
var enumerator = rows.WithCancellation(ct).GetAsyncEnumerator();
StreamWriter? writer = null;
var written = 0;
try
{
// Read the first row before writing the HTTP response body. This lets an
// initial database error be handled before any CSV bytes are committed.
var hasFirstRow = await enumerator.MoveNextAsync();
// Format the first row before writing the header so an unexpected formatting
// failure cannot leave a header buffered for disposal to send by itself.
var firstCsvRow = hasFirstRow
? FormatRow(enumerator.Current)
: null;
// Do not create the BOM-enabled writer until the first query succeeds.
writer = CreateWriter(output);
await writer.WriteLineAsync(Header.AsMemory(), ct);
if (firstCsvRow is not null)
{
await writer.WriteLineAsync(firstCsvRow.AsMemory(), ct);
written++;
}
// Commit the header and first row promptly; subsequent writes are batched.
await writer.FlushAsync(ct);
while (await enumerator.MoveNextAsync())
{
await writer.WriteLineAsync(
FormatRow(enumerator.Current).AsMemory(), ct);
written++;
if (written % FlushEveryRows == 0)
await writer.FlushAsync(ct);
}
await writer.FlushAsync(ct);
logger.LogInformation(
"CSV order export completed. Rows={Rows}, Batched={Batched}",
written, useBatches);
}
catch (OperationCanceledException) when (ct.IsCancellationRequested)
{
// RequestAborted can be triggered by the client or by the server.
logger.LogInformation(
"CSV order export cancelled after {Rows} rows.",
written);
}
catch (IOException ex) when (ct.IsCancellationRequested)
{
// A closed response can surface as an I/O error instead of cancellation.
// If your provider reports cancellation through its own documented
// exception type, handle that type explicitly here too.
logger.LogInformation(ex,
"CSV response stream ended after the request was aborted at {Rows} rows.",
written);
}
catch (Exception ex)
{
logger.LogError(ex,
"CSV order export failed after {Rows} rows.", written);
throw;
}
finally
{
try
{
try
{
await enumerator.DisposeAsync();
}
catch (Exception ex) when (ct.IsCancellationRequested)
{
logger.LogDebug(ex, "The export reader reported an error while closing after cancellation.");
}
}
finally
{
if (writer is not null)
await DisposeWriterQuietlyAsync(writer, ct);
}
}
}
private IQueryable<Order> Filter(DateTime fromUtc, DateTime toUtc) =>
db.Orders
.AsNoTracking()
.Where(o => o.CreatedUtc >= fromUtc && o.CreatedUtc < toUtc);
private static IQueryable<OrderRow> ToRows(IQueryable<Order> orders) =>
orders.Select(o => new OrderRow(
o.Id,
o.CustomerName,
o.Email,
o.Total,
o.Status,
o.CreatedUtc));
// Keyset pagination reads a bounded batch, completes that query, and only
// then yields rows. Id is a unique, stable primary-key cursor.
private async IAsyncEnumerable<OrderRow> ReadInBatches(
DateTime fromUtc,
DateTime toUtc,
[EnumeratorCancellation] CancellationToken ct)
{
int? lastId = null;
while (true)
{
var query = Filter(fromUtc, toUtc);
if (lastId is int afterId)
query = query.Where(o => o.Id > afterId);
var batch = await ToRows(query.OrderBy(o => o.Id))
.Take(BatchSize)
.ToListAsync(ct);
if (batch.Count == 0)
yield break;
foreach (var row in batch)
yield return row;
lastId = batch[^1].Id;
}
}
private static StreamWriter CreateWriter(Stream output) =>
new(
output,
new UTF8Encoding(encoderShouldEmitUTF8Identifier: true),
bufferSize: 32 * 1024,
leaveOpen: true)
{
NewLine = "\r\n"
};
private static async Task DisposeWriterQuietlyAsync(
StreamWriter writer,
CancellationToken ct)
{
try
{
await writer.DisposeAsync();
}
catch (Exception ex) when (
ct.IsCancellationRequested &&
(ex is OperationCanceledException or IOException or ObjectDisposedException))
{
// The response stream may already be closed. Do not hide other dispose errors.
}
}
private static string FormatRow(OrderRow order) =>
string.Join(',',
order.Id.ToString(Invariant),
CsvField.Escape(order.CustomerName),
CsvField.Escape(order.Email),
order.Total.ToString("F2", Invariant),
CsvField.Escape(order.Status),
// CreatedUtc is stored by application convention as UTC. datetime2
// itself does not record a time-zone offset. Seven digits retain ticks.
order.CreatedUtc.ToString("yyyy-MM-dd'T'HH:mm:ss.fffffff'Z'", Invariant));
}
How the exporter works
-
AsNoTracking(): makes the read-only intent explicit. The query projects into a record of scalar values, so EF Core wouldn't track entity instances here anyway. -
AsAsyncEnumerable(): exposes the query as an asynchronous sequence, so the application consumes rows one at a time instead of callingToListAsync()on the full result. -
WithCancellation(ct): passes the request token to the enumerator. TheWriteLineAsync(ReadOnlyMemory<char>, CancellationToken)andFlushAsync(ct)overloads let response writes stop when the request is cancelled. - First-row prefetch: catches failures from the initial query before the response body starts. Later failures can still interrupt a download that has already begun.
-
Projection:
OrderRowholds only the fields written to the CSV. Don't touch navigation properties or trigger lazy loading inside the loop. -
Invariant formatting: decimals always use a dot, and
timestamps use a fixed format. The literal
Zis correct only because the sample stores UTC by convention. -
Cancellation handling: only the expected cases are treated
as a normal stop: an
OperationCanceledExceptionor anIOExceptionwhile the request token is cancelled. Remember thatRequestAbortedcan fire for server-side reasons as well as a client disconnect. Every other exception is logged as an error and rethrown, so an unrelated failure that happens near a disconnect isn't hidden. If your provider reports cancellation through its own exception type, handle that documented type explicitly. - Command timeout: the sample sets five minutes for this scoped context. That's an example value, not a recommendation, and it doesn't replace HTTP, proxy, or client timeouts.
- Asynchronous disposal: the writer leaves the framework-owned response stream open and is disposed asynchronously so cleanup never performs a synchronous response write.
6. Register services and map the minimal API endpoint
Program.cs registers the SQL Server context,
the exporter, a rate limiter, and an authorization policy named
OrdersExport. In Development the policy
allows anonymous requests, so you can follow the tutorial without an identity
provider. In every other environment it requires a JWT with the example claim
permission=orders.export. Outside Development
it also registers the exception handler, so a failure before the first
response byte (such as the initial query failing) returns a Problem Details
500 instead of an empty response. In a minimal API, the
CancellationToken parameter is bound to
HttpContext.RequestAborted; see the
minimal API parameter-binding reference.
The response is created with the callback overload of
Results.Stream. Microsoft documents that the
Results.Stream overloads give you access to
the underlying response stream without buffering it first, which is why the
exporter can write rows as they arrive. See the
minimal API responses documentation.
// Program.cs
using Codingvila.CsvExport.Data;
using Codingvila.CsvExport.Services;
using Microsoft.AspNetCore.Authentication.JwtBearer;
using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.RateLimiting;
using Microsoft.EntityFrameworkCore;
var builder = WebApplication.CreateBuilder(args);
var connectionString = builder.Configuration.GetConnectionString("Default")
?? throw new InvalidOperationException(
"Connection string 'Default' was not configured.");
builder.Services.AddDbContext<AppDbContext>(options =>
options.UseSqlServer(connectionString));
builder.Services.AddScoped<OrderCsvExporter>();
builder.Services.AddProblemDetails();
// The limiter is per application process. Use an API gateway or a distributed
// limiter if you need one shared limit across multiple app instances.
builder.Services.AddRateLimiter(options =>
{
options.RejectionStatusCode = StatusCodes.Status429TooManyRequests;
options.AddConcurrencyLimiter("export", limiter =>
{
limiter.PermitLimit = 2;
limiter.QueueLimit = 0;
});
});
// The policy exists in every environment so controllers and endpoints can
// reference it. Development allows anonymous access for the tutorial only.
builder.Services.AddAuthorization(options =>
{
options.AddPolicy("OrdersExport", policy =>
{
if (builder.Environment.IsDevelopment())
{
policy.RequireAssertion(_ => true);
}
else
{
// Example claim contract. Map your identity provider's export permission
// to this claim, or replace it with your provider's scope/role policy.
policy.RequireAuthenticatedUser()
.RequireClaim("permission", "orders.export");
}
});
});
if (!builder.Environment.IsDevelopment())
{
var authority = builder.Configuration["Authentication:Authority"];
var audience = builder.Configuration["Authentication:Audience"];
if (string.IsNullOrWhiteSpace(authority) || string.IsNullOrWhiteSpace(audience))
throw new InvalidOperationException(
"Authentication:Authority and Authentication:Audience must be configured outside Development.");
builder.Services
.AddAuthentication(JwtBearerDefaults.AuthenticationScheme)
.AddJwtBearer(options =>
{
options.Authority = authority;
options.Audience = audience;
});
}
var app = builder.Build();
// Development already uses the developer exception page. Elsewhere, return
// Problem Details for failures that happen before the first CSV byte.
if (!app.Environment.IsDevelopment())
app.UseExceptionHandler();
// Endpoint-specific policies and rate limits need routing to run first.
app.UseRouting();
if (!app.Environment.IsDevelopment())
app.UseAuthentication();
app.UseAuthorization();
// Rate limiting runs after authorization so unauthenticated requests do not
// occupy the export concurrency permits.
app.UseRateLimiter();
const int maxRangeDays = 366;
app.MapGet("/api/orders/export", (
[FromQuery(Name = "from")] DateTimeOffset? fromDate,
[FromQuery(Name = "to")] DateTimeOffset? toDate,
bool? batched,
HttpContext http,
OrderCsvExporter exporter,
CancellationToken ct) =>
{
var toUtc = toDate?.UtcDateTime ?? DateTime.UtcNow;
var fromUtc = fromDate?.UtcDateTime ?? toUtc.AddDays(-30);
// Validate before the first response byte. Once streaming starts, a clean
// Problem Details response can no longer replace the CSV body.
if (fromUtc >= toUtc || toUtc - fromUtc > TimeSpan.FromDays(maxRangeDays))
{
return Results.Problem(
detail: $"'from' must be earlier than 'to', and the range cannot exceed {maxRangeDays} days.",
statusCode: StatusCodes.Status400BadRequest);
}
http.Response.Headers.CacheControl = "no-store";
http.Response.Headers["X-Content-Type-Options"] = "nosniff";
var fileName = $"codingvila-orders-{DateTime.UtcNow:yyyyMMdd-HHmmss}.csv";
return Results.Stream(
stream => exporter.WriteAsync(
stream, fromUtc, toUtc, batched ?? false, ct),
contentType: "text/csv; charset=utf-8",
fileDownloadName: fileName);
})
.RequireAuthorization("OrdersExport")
.RequireRateLimiting("export");
app.Run();
Outside Development, supply
Authentication:Authority and
Authentication:Audience through environment
variables or your secret store. The JSON below shows the shape; the values are
placeholders:
{
"Authentication": {
"Authority": "https://login.example.com/",
"Audience": "codingvila-orders-api"
}
}
Map your identity provider's real scope or role claim to the policy you use.
The sample table is single-tenant. In a multi-tenant app, add a tenant key to
the entity and apply the tenant predicate in
Filter, deriving the tenant from the
authenticated user rather than from an unrestricted query-string value.
Authentication alone doesn't make a cross-tenant query safe.
Development-only warning: the Development policy deliberately
allows anonymous export requests. Don't deploy this sample with
ASPNETCORE_ENVIRONMENT=Development, and don't
put real customer data in the development database.
The limiter allows two concurrent exports per application process and answers extra requests with HTTP 429. ASP.NET Core's built-in limiter keeps its counters in process memory, so three instances can admit six exports in total. Use an API gateway or a distributed limiter when you need one deployment-wide cap. See Microsoft's ASP.NET Core rate-limiting documentation before choosing a production policy.
Now build the project, create the migration, and apply it. The
-- --environment Development part matters:
dotnet ef runs your
Program.cs in the Production environment by
default, and the sample would stop with a missing-authentication-settings
error.
dotnet build
dotnet ef migrations add InitialCreate -- --environment Development
dotnet ef database update -- --environment Development
Expected result:
dotnet ef database update finishes with
Done. and creates the
CodingvilaCsvExport database with an
Orders table.
Run the API
Start the app with the HTTPS launch profile. The console prints the URL it
listens on; use that address in the examples below. If your browser or curl
reports a certificate error on Windows or macOS, run
dotnet dev-certs https --trust once. On
Linux, trusting the development certificate depends on your distribution and
browser, so follow the Linux instructions in Microsoft's HTTPS development
certificate documentation.
dotnet run --launch-profile https
How to call the endpoint
Without query parameters, the endpoint exports the previous 30 days. The
from boundary is inclusive and
to is exclusive. Both accept ISO 8601
date-times with an offset, for example:
/api/orders/export?from=2026-01-01T00:00:00Z&to=2026-02-01T00:00:00Z
Always include Z or an explicit offset. A
value such as 2026-01-01T00:00:00 has no
offset, so ASP.NET Core reads it in the server's local time zone and your
range can shift silently. If you send an offset like
+05:30, URL-encode the plus sign as
%2B; an unescaped plus in a query string is
commonly decoded as a space. The endpoint converts both boundaries to UTC
before querying. A malformed value is rejected during parameter binding, and a
reversed or overlong range returns HTTP 400 before streaming begins. Add
&batched=true to try the keyset-batched
mode from section 8.
Once you've seeded the database (section 9), the export lists orders in
ascending Id order, and the special edge-case
rows come last because they were inserted after the generated ones. The
examples below illustrate individual record formats; they aren't consecutive
lines from the file. The generated row assumes a freshly created table, where
order number 999999 gets Id 999999 and a
total of 499.99. The timestamp is a fixed example to show the seven-digit UTC
format; the seed script generates a different timestamp each time it runs.
Id,CustomerName,Email,Total,Status,CreatedUtc
999999,Customer 999999,customer999999@example.com,499.99,New,2025-06-15T10:15:30.1234567Z
For example, the formula-like customer name
=HYPERLINK("http://example.com") is prefixed
and escaped by CsvField.Escape, and appears
in the comma-delimited output like this:
"'=HYPERLINK(""http://example.com"")"
7. Use an MVC controller instead
If your app uses MVC controllers, call the same exporter and write to
Response.Body. Register controllers with
builder.Services.AddControllers(), map them
with app.MapControllers(), and remove the
minimal API route so both aren't mapped to the same path. The
OrdersExport policy from section 6 exists in
every environment, so the attribute below works in Development too.
// Controllers/OrdersController.cs
using Codingvila.CsvExport.Services;
using Microsoft.AspNetCore.Authorization;
using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.RateLimiting;
[ApiController]
[Authorize("OrdersExport")]
[Route("api/orders")]
public sealed class OrdersController(OrderCsvExporter exporter) : ControllerBase
{
[HttpGet("export")]
[EnableRateLimiting("export")]
public async Task<IActionResult> ExportAsync(
[FromQuery(Name = "from")] DateTimeOffset? fromDate,
[FromQuery(Name = "to")] DateTimeOffset? toDate,
CancellationToken ct)
{
var toUtc = toDate?.UtcDateTime ?? DateTime.UtcNow;
var fromUtc = fromDate?.UtcDateTime ?? toUtc.AddDays(-30);
if (fromUtc >= toUtc || toUtc - fromUtc > TimeSpan.FromDays(366))
{
return Problem(
detail: "'from' must be earlier than 'to', and the range cannot exceed 366 days.",
statusCode: StatusCodes.Status400BadRequest);
}
Response.ContentType = "text/csv; charset=utf-8";
Response.Headers.CacheControl = "no-store";
Response.Headers["X-Content-Type-Options"] = "nosniff";
Response.Headers["Content-Disposition"] =
$"attachment; filename=\"codingvila-orders-{DateTime.UtcNow:yyyyMMdd-HHmmss}.csv\"";
await exporter.WriteAsync(
Response.Body, fromUtc, toUtc, useBatches: false, ct);
return new EmptyResult();
}
}
Keep the same routing, authentication, authorization, and rate-limiting
middleware from Program.cs. The action's
CancellationToken is bound to the current
request. The controller repeats the date validation; in a larger codebase,
move it into one small shared service so the rules live in a single place.
8. Use keyset batching when a single reader is too long-lived
A single AsAsyncEnumerable() query normally
keeps its database reader and connection active until enumeration finishes, so
a slow client can occupy a pooled SQL connection for most of the download. The
optional ReadInBatches method instead fetches
up to 2,000 rows where Id is greater than the
last seen id, completes that query, and then yields the materialized batch to
the writer. The reader is closed between batches.
Keyset pagination avoids the growing skip cost of large
Skip/Take offsets, but it isn't automatically
faster for every date-filtered query. A keyset cursor must use a unique,
stable ordering. Here, Id is the table's
primary key, so the predicate
Id > lastId can move forward without
assuming that ids start at 1 or are gap-free. If you sort by a non-unique
field such as a timestamp, include a unique tie-breaker and carry both values
in the cursor. The sample orders by Id and
filters separately on CreatedUtc. If only a
small fraction of rows match the date filter, a date index plus a sort may be
cheaper; if most rows match, scanning in primary-key order may win. Check the
execution plan on production-like data.
Batching also doesn't make the HTTP download resumable. If the client disconnects, the next request starts a new export unless you build a checkpoint, snapshot, or stored-file workflow.
Batching bounds application memory to one batch plus writer and transport buffers, but it does not give you a point-in-time snapshot. Rows inserted or changed between batch queries can appear, vanish, or show a different state. What a single query sees depends on the database's isolation configuration.
With lock-based READ COMMITTED (READ_COMMITTED_SNAPSHOT
OFF), concurrent changes can affect the rows a long-running query observes as
it progresses, so it isn't a stable point-in-time view of the export. When
READ_COMMITTED_SNAPSHOT is ON (the default in
Azure SQL Database), each SELECT statement
sees a statement-level snapshot. A single-reader export therefore sees one
statement-level snapshot, while separate keyset-batch queries can each see a
different one.
For one transaction-level view across all batches, use
SNAPSHOT isolation, which requires enabling
the database's
ALLOW_SNAPSHOT_ISOLATION option first. A
long-running snapshot transaction keeps the connection open across batches and
retains row versions until it completes, which reduces the connection-release
benefit of batching and can increase version-store pressure. Monitor
transaction duration and version-store usage before adopting this design.
9. Seed SQL Server and test the export
Insert one million sample orders
After applying the migration, run this script against the
CodingvilaCsvExport database in SQL Server
Management Studio, Azure Data Studio, or
sqlcmd. A small tally CTE (so it doesn't
depend on system catalog row counts) generates one million orders, one every
25 seconds, covering roughly the previous 289 days with the newest order
getting the highest id. After that come test rows containing commas, quotes,
the common formula-triggering prefixes, and embedded line breaks.
WITH
E1(n) AS (SELECT 1 UNION ALL SELECT 1), -- 2 rows
E2(n) AS (SELECT 1 FROM E1 AS a CROSS JOIN E1 AS b), -- 4 rows
E4(n) AS (SELECT 1 FROM E2 AS a CROSS JOIN E2 AS b), -- 16 rows
E8(n) AS (SELECT 1 FROM E4 AS a CROSS JOIN E4 AS b), -- 256 rows
E16(n) AS (SELECT 1 FROM E8 AS a CROSS JOIN E8 AS b), -- 65,536 rows
E20(n) AS (SELECT 1 FROM E16 AS a CROSS JOIN E4 AS b), -- 1,048,576 rows
nums AS
(
SELECT TOP (1000000)
ROW_NUMBER() OVER (ORDER BY (SELECT NULL)) AS n
FROM E20
)
INSERT INTO Orders (CustomerName, Email, Total, Status, CreatedUtc)
SELECT
CONCAT('Customer ', n),
CONCAT('customer', n, '@example.com'),
(n % 50000) / 100.0,
'New',
-- One order every 25 seconds. The highest n is the newest order, so Id
-- and CreatedUtc increase together.
DATEADD(SECOND, CAST((n - 1000000) * 25 AS int), SYSUTCDATETIME())
FROM nums
ORDER BY n; -- makes identity values follow n
-- These rows cover delimiters, quotes, multiline text, and formula-triggering prefixes.
INSERT INTO Orders (CustomerName, Email, Total, Status, CreatedUtc)
VALUES
('Doe, Jane', 'jane@example.com', 19.99, 'New', SYSUTCDATETIME()),
('=HYPERLINK("http://example.com")', NULL, 1.00, 'New', SYSUTCDATETIME()),
('+1+1', NULL, 2.00, 'New', SYSUTCDATETIME()),
('-1+1', NULL, 3.00, 'New', SYSUTCDATETIME()),
('@SUM(1,1)', NULL, 4.00, 'New', SYSUTCDATETIME()),
(CHAR(9) + '=1+1', NULL, 5.00, 'New', SYSUTCDATETIME()),
(CHAR(13) + '=1+1', NULL, 6.00, 'New', SYSUTCDATETIME()),
(CHAR(10) + '=1+1', NULL, 7.00, 'New', SYSUTCDATETIME()),
(N'=1+1', NULL, 8.00, 'New', SYSUTCDATETIME()),
(N'+1+1', NULL, 9.00, 'New', SYSUTCDATETIME()),
(N'-1+1', NULL, 10.00, 'New', SYSUTCDATETIME()),
(N'@SUM(1,1)', NULL, 11.00, 'New', SYSUTCDATETIME()),
('Line one' + CHAR(13) + CHAR(10) + 'Line two "quoted"', NULL, 5.00, 'Paid', SYSUTCDATETIME());
With no parameters, the default 30-day range returns roughly 104,000 generated rows plus the special test rows. The 366-day limit isn't a row-count limit: real workloads can hold far more rows in the same period, so define a workload-specific row quota or send very large exports through a background job.
Run the seed script once, against an empty development database; running it again inserts another million generated orders and another set of test rows. The 104,000-row estimate also assumes you run the export shortly after seeding, because the timestamps are anchored to the moment the script runs.
Download the CSV with curl
Set the base URL to the HTTPS address printed by
dotnet run. Trust the ASP.NET Core
development certificate instead of using
-k as a routine workaround. Expected result:
orders.csv appears in the current folder and
starts with the header line
Id,CustomerName,Email,Total,Status,CreatedUtc.
# Replace the URL with the HTTPS address printed by dotnet run.
BASE_URL="https://localhost:7001"
# GNU/Linux date; on macOS use: date -u -v-120d +%Y-%m-%dT%H:%M:%SZ
FROM=$(date -u -d '120 days ago' +%Y-%m-%dT%H:%M:%SZ)
curl --fail --show-error --output orders.csv "$BASE_URL/api/orders/export?from=$FROM"
On Windows PowerShell, use curl.exe, because
curl can be an alias for a different
PowerShell command. The "o" format string
produces an invariant ISO 8601 UTC timestamp when the value's kind is UTC:
# Replace the URL with the HTTPS address printed by dotnet run.
$baseUrl = "https://localhost:7001"
$from = [DateTime]::UtcNow.AddDays(-120).ToString("o")
curl.exe --fail --show-error --output orders.csv "$baseUrl/api/orders/export?from=$from"
To simulate a slow client, add
--limit-rate 8k to the curl command. The
response should start before the full query has been consumed, subject to
buffering in the hosting stack and any proxy in front of it.
Check memory usage
Install the .NET counters tool and monitor the running process. Replace
12345 with the process ID from your operating
system or dotnet-counters ps:
dotnet tool install --global dotnet-counters
dotnet-counters monitor --process-id 12345 --counters System.Runtime
Compare the garbage-collected heap while exporting the same data with a fully buffered implementation and with this streaming one. The streamed version should avoid memory growth proportional to the whole result set, although the heap will still move as batches, strings, and buffers are allocated.
Test cancellation
Start a large download with curl and press Ctrl+C in the terminal where curl
is running; that closes the client connection while the server is still
writing. The application log should report that the export was cancelled or
that the response stream ended after the request was aborted. Then inspect
active SQL requests. On SQL Server 2019 and earlier, viewing all sessions
requires VIEW SERVER STATE; SQL Server 2022
and later require
VIEW SERVER PERFORMANCE STATE. Microsoft
documents that, without that permission, you see only your own session.
Permissions differ on Azure SQL, so check the
sys.dm_exec_requests documentation
for your platform.
SELECT
r.session_id,
r.status,
r.command,
txt.text
FROM sys.dm_exec_requests AS r
CROSS APPLY sys.dm_exec_sql_text(r.sql_handle) AS txt
WHERE txt.text LIKE '%Orders%'
AND r.session_id <> @@SPID;
Don't treat a partially downloaded CSV as complete; delete it and repeat the export. EF Core passes cancellation tokens to the provider, but the provider may not honor cancellation instantly, so verify the behavior in your environment.
Validate CSV content in a spreadsheet
-
Confirm that
Doe, Janestays in one field. - Confirm that double quotes inside the multiline value are doubled in the raw CSV and display correctly in the spreadsheet.
- Confirm that formula test values appear as text and aren't evaluated. The leading apostrophe may stay visible.
- Check non-ASCII customer names to verify UTF-8 decoding.
- Inspect the raw file as text. A spreadsheet's display doesn't replace checking the actual bytes and field boundaries.
Each order should occupy one CSV record even when a text field contains a quoted line break. A successful HTTP status alone doesn't prove that a long streamed file arrived completely.
10. Production limitations and deployment considerations
Errors after streaming starts cannot become a clean HTTP 500
Once response bytes are sent, the status code and headers can't be replaced by a Problem Details response. If a later database or serialization error occurs, the client can receive a truncated CSV that began with HTTP 200. Log the exception with a correlation ID and consider whether consumers need an out-of-band completion check.
For financial, regulatory, or other all-or-nothing exports, generate the file in temporary or object storage first and publish a download link only after it succeeds. That delays the download but supports retry, retention, status reporting, and integrity checks far better than a live response.
Know when to use a background export job
Direct streaming fits when the query and transfer finish within the limits of your server, reverse proxy, and client. Use a background job when exports are very large, outlast gateway timeouts, must be resumable, or must arrive as a complete artifact. A common design accepts the request, returns HTTP 202 Accepted with a job-status URL, writes the file to object storage in a worker, and exposes a short-lived download URL when the job succeeds. Queue, storage, retention, and authorization details depend on your hosting environment. In short: if the file must be complete, must be resumable, or may outlast your gateway timeouts, use a background job that writes a stored file; otherwise, stream directly. The flowchart below shows the same decision, and the table after it adds more criteria.
| Requirement | Choose direct streaming | Choose background generation |
|---|---|---|
| Download should start while rows are read | Good fit | Not the primary benefit |
| File must be complete before user receives it | Not a good fit | Good fit |
| Gateway timeout may be exceeded | Risky | Good fit |
| Resume or retry a failed download | Limited | Can be designed around stored files and range requests |
| Small or moderate one-off export | Usually simpler | May add unnecessary infrastructure |
Date windows are not row-count limits
A busy tenant can have millions of rows in a short period, so a date limit doesn't cap the export size. Set a business-appropriate row quota based on measured capacity. When you must reject oversized exports before sending any bytes, use a precomputed count, a materialized snapshot, or a background job. Avoid silently stopping a live CSV at an arbitrary row and presenting it as complete.
Database connections and DbContext lifetime
During a single-reader export, EF Core keeps reading from the database while
the response is written, so a slow client can hold the connection. Limit
concurrent exports, monitor connection-pool usage, and never run another
operation on the same DbContext while its
query is being enumerated; EF Core doesn't support parallel operations on one
context instance. A transaction that spans the whole export keeps connection
and transaction resources open even in batched mode, so choose batching and
isolation together.
EF Core may buffer internally
AsAsyncEnumerable() allows application-level
streaming, but EF Core doesn't always stream. A retrying execution strategy is
a well-known case that makes the provider buffer results so a failed query can
be retried, and some split-query setups can buffer as well. This sample uses
one projected query and doesn't enable a retrying execution strategy, but
review your own configuration before promising constant memory. Microsoft's
efficient querying guide for EF Core
covers projection, tracking, and related trade-offs.
Kestrel minimum response data rate
Kestrel documents a default minimum response data rate of 240 bytes per second
with a five-second grace period. If a client reads too slowly, the connection
can be timed out. The per-request
IHttpMinResponseDataRateFeature is not
available for HTTP/2; configure the server-wide
KestrelServerOptions.Limits.MinResponseDataRate
when a server-wide change is justified. Microsoft also notes that
operating-system TCP buffering in HTTP/2 and QUIC buffering in HTTP/3 can make
slow-reader detection less immediate. Don't disable the protection globally
just to make one export work. Reproduce the issue on your actual protocol and
hosting path, then tune only the layer that measurements show is responsible.
See the
Kestrel options documentation.
Reverse-proxy buffering and response compression
Application-level streaming doesn't guarantee that every proxy forwards bytes
immediately. Nginx enables proxy response buffering by default, which can hold
response data in memory and temporary files. For a dedicated streaming route,
set proxy_buffering off in the matching Nginx
location, or send X-Accel-Buffering: no if
the proxy honors it (the proxy can be configured to ignore the header). See
the
Nginx proxy buffering documentation. Response compression can also change when data reaches the client. Test the
complete path through your proxy, gateway, CDN, and hosting platform.
If you enable ASP.NET Core response compression for the export, check that
text/csv is covered by your compression
configuration, because compression is applied by MIME type. A compressor and
any proxy in front of it keep their own buffers, so the first bytes may reach
the client later than in the uncompressed case. Verify the behavior with
curl --compressed and a slow client on your
real deployment path, or compress at the proxy or gateway instead and exclude
this route if early delivery matters.
No Content-Length and download progress
A streamed response normally doesn't know its final
Content-Length when it starts. HTTP/1.1
commonly uses chunked transfer encoding in that case, while HTTP/2 and HTTP/3
frame the response with their own protocols. The browser may show an
indeterminate download rather than a percentage. Don't invent a progress
percentage without a trustworthy total size or row count.
Browser-side downloads can reintroduce buffering
A normal browser navigation to a download URL lets the browser handle the
response as a file. JavaScript that calls
response.blob() or
response.arrayBuffer() consumes the complete
response body before the promise resolves and gives your code a
Blob or
ArrayBuffer. A browser may back a
Blob with memory or temporary storage, so it
is too broad to say every Blob is kept entirely in RAM; either approach can
still create substantial client-side resource pressure for a very large
export. Prefer a normal download navigation or a client that processes a
ReadableStream incrementally.
If your API uses bearer tokens and a normal navigation can't send the Authorization header, use a short-lived, single-use download ticket or a streaming-capable client. Never put a long-lived access token or JWT in a query string: URLs end up in server logs, browser history, monitoring systems, and referrer data.
Excel's worksheet row limit
A modern Excel worksheet holds 1,048,576 rows, including the header. A CSV can contain more records, but Excel won't show them all in one worksheet. Split the export, use a database or reporting tool, or choose another format when consumers need more rows in one workbook.
Command timeout is not an end-to-end download timeout
SqlCommand.CommandTimeout defaults to 30
seconds, and the sample raises it to five minutes for commands issued by this
export's scoped context. With the SQL Server provider, the timeout counts
network-read time during command execution and result processing; it can occur
after the first row is returned, and time spent processing a row in your
application isn't counted. The timeout is applied to a network-read method
invocation, not as one overall wall-clock limit for the entire CSV download.
It doesn't configure the reverse proxy, Kestrel, browser, or client timeout.
See Microsoft's
SqlCommand.CommandTimeout documentation
and tune each layer from measured durations.
CSV compatibility and regional delimiters
The sample emits comma-separated fields and a UTF-8 BOM. Some regional spreadsheet settings expect semicolons, so make the delimiter an explicit export option if users need more than one dialect rather than changing it for everyone. A BOM helps many desktop spreadsheets detect UTF-8, but some machine consumers expect no BOM. Document the file contract.
11. Common errors and troubleshooting
Authentication settings error when running
dotnet ef
If dotnet ef complains that
Authentication:Authority and
Authentication:Audience must be configured,
the tool ran your app in the Production environment. Add
-- --environment Development, as shown in
section 6.
Unable to create a DbContext when running
dotnet ef
Confirm that
AddDbContext<AppDbContext> is
registered in Program.cs, the connection
string key is Default, the EF Core Design
package is installed, and
dotnet build succeeds before you run
migrations.
Synchronous operations are disallowed
Something in the response path is writing or flushing synchronously. Use the
cancellation-aware WriteLineAsync and
FlushAsync overloads, avoid synchronous
disposal that flushes the response, and don't enable
AllowSynchronousIO as a workaround.
There is already an open DataReader, or a second operation was started on this context
The app ran another query on a
DbContext while the export reader was active.
Depending on the provider and configuration, you may see an "already an open
DataReader" message or EF Core's "second operation was started on this context
instance" message. Project every needed value before the stream starts, avoid
lazy loading, and don't run parallel operations on the same context. Use a
second context, or batched mode, only when the extra query is truly required.
The CSV is garbled in Excel
Check that the file is UTF-8 and that the writer emits the BOM. Then inspect the raw bytes and confirm that no middleware or downstream step changed the encoding. For machine-to-machine imports, check whether the consumer expects a BOM.
The download stops partway through
Check the application log for the export exception, then look at command timeout, Kestrel limits, proxy buffering, gateway idle timeouts, and client disconnects. A response that has started can't be replaced by a clean error document. If complete delivery and retry are requirements, generate the file in the background.
The endpoint returns HTTP 429
The concurrency limiter has no free permits. Wait for an active export to finish, or change the limit only after measuring database, memory, and connection-pool capacity. The built-in in-memory limiter applies per application instance.
The date range looks shifted
The from or
to value probably had no offset, so the
server read it as local time. Add Z or an
explicit offset.
The export is slow even though the response is streamed
Streaming controls how results are consumed and sent; it doesn't make a slow SQL query fast. Check the execution plan, index usage, rows read versus rows returned, sort operators, blocking, and network throughput. A read replica can offload the query, but it isn't a point-in-time consistency guarantee by itself.
12. Frequently asked questions
How do I export a large CSV file in ASP.NET Core without loading everything into memory?
Write directly to the response stream with
Results.Stream or
Response.Body, and feed the writer from an EF
Core query consumed through
AsAsyncEnumerable(). Don't materialize the
full result set, and avoid making a browser client buffer the complete
response with response.blob() or
response.arrayBuffer() when files are very
large. See the
ASP.NET Core 10 Results.Stream API reference.
Does EF Core's AsAsyncEnumerable() guarantee constant memory usage?
No. It streams results to your code in the normal case, but EF Core can buffer internally in some configurations, such as with a retrying execution strategy. The provider, server, proxy, and client can also keep their own buffers. Review your actual EF Core configuration and deployment path before describing memory use as constant.
How do I cancel an export when the user closes the browser?
Accept a CancellationToken in the minimal API
or controller action and pass it through the enumerator, database calls,
writes, and flushes. In a minimal API the token is bound to
HttpContext.RequestAborted. The database
provider may not stop the command instantly, so verify cancellation in your
environment.
Should I use AsAsyncEnumerable() or keyset pagination?
Use a single async enumeration when a long-lived reader is acceptable and one
query is the simplest fit. Use keyset batches when releasing the reader
between shorter queries matters more. The cursor must be unique and stable;
this sample uses the Id primary key. Batches
add queries and don't give you one consistent snapshot unless you introduce an
explicit consistency strategy.
How do I make Excel recognize UTF-8 CSV?
Emit UTF-8 with a byte order mark, as this sample does with
UTF8Encoding(encoderShouldEmitUTF8Identifier: true). Test the file in the spreadsheet versions your users actually have.
How do I protect a CSV export from formula injection?
Escape delimiters and quotes correctly, detect text cells that begin with formula-triggering characters, and apply a mitigation chosen for your spreadsheet workflow. OWASP documents apostrophe-prefix handling but warns that Excel may remove protective characters when a file is saved and reopened. A tab prefix is an Excel-oriented alternative, but it remains in the data and can affect later imports. There is no universal mitigation for every spreadsheet and downstream parser, so test the exact applications and file workflow you support.
Can I use CsvHelper to stream an EF Core query?
Yes. CsvHelper supports asynchronous writing, including asynchronous record writing. Use its async APIs with the response stream, pass cancellation through, and test with the version you install. The same limits around cancellation, proxy buffering, and partial downloads still apply.
When should I use a background export instead of streaming?
Use a background job when an export may exceed gateway or client timeouts, must be retryable or resumable, needs all-or-nothing delivery, or is expensive enough to schedule and monitor separately. Serve the finished file from durable storage after the job succeeds.
Key takeaways
- Stream the query and the response together, and never materialize the whole result or the whole CSV.
- Validate input and fetch the first row before any bytes go out, because later errors can only truncate the download.
- Quote CSV correctly and treat formula injection as a separate security concern with no universal fix.
- Limit concurrent exports and watch database connections, proxy buffering, and timeouts across every layer.
- Switch to a background job when the file must be complete, resumable, or independent of one HTTP request.
Conclusion
To stream large CSV exports in ASP.NET Core with EF Core, stream the query and the HTTP response together: project only the columns you need, enumerate asynchronously, write with cancellation-aware APIs, and flush at deliberate intervals. Escape CSV text correctly, treat formula injection as its own security problem, validate the date range before the first response byte, and limit concurrent exports.
A single async reader is usually the simplest design, but it holds a database connection while the client downloads. Keyset batches shorten each reader's lifetime at the cost of extra queries and weaker consistency across batches. Neither approach makes an export snapshot-consistent or immune to proxy buffering and client timeouts. When the file must be complete, resumable, or independent of one HTTP request's lifetime, generate it as a background artifact and serve it after it succeeds.
