File attachments

Xata offers general purpose file attachment capabilities. In order to simplify the management of large binary objects and to improve the developer experience, Xata integrates files support directly into the database itself. The aim is to offer a unified, seamless experience by exposing file support under one API, security model, and SDK. Using this approach, the queries, filters, summaries, aggregations from the SDK can also be used on the files' metadata. The metadata (file name, media type, tags) is also indexed for search so it can be included in search results.

To support file attachments, Xata provides two column types:

• file column type
  • The column stores file objects, which are JSON objects with a pre-defined schema.
  • Since the column stores a single file object, the file metadata fields can be used in query filters or in summary queries.
  • Files stored in this column type do not get a file id assigned, as they can be referenced by the record id and column name.
• file[] (file array) column type
  • The column stores an array of file item objects.
  • As the column may store an arbitrary number of files, this content cannot be used in query filters or in summary queries.
  • Files stored in this array column type carry a file id so they can be referenced in endpoints such as the binary File API.

A full breakdown of the file and file[] column types can be found on the data model page.

File attachments are delivered through an integrated content delivery network (CDN) which minimizes access latency by using regional caches.

Caching is enabled for all signed and public URLs. This is to avoid caching data that is secured by API key permissions. This feature is enabled by default and doesn't require any configuration. Content from publicly accessible and signed URLs is cached for a duration of 4 hours.

As with all cache systems, the fundamental problem is cache invalidation - maintaining cache access performance while ensuring updated content and preventing outdated entries. In its architectural approach, Xata addresses the issue of cache invalidation by treating file content as dynamic - analogous to data within a database.

Consequently, it's recommended that clients dynamically obtain access URLs from the database, rather than persisting them in static resources. When the target file is modified, URLs can become invalid, resulting in access disruptions if stored externally to the database.

Through the process of generating a new URL after each update, the system prevents the delivery of outdated and stale content, regardless of any caches or proxies between the client and storage service. For the client application, this translates to no need for cache time-to-live (TTL) or waiting for cache invalidation. When the URL is fetched directly from the database, it is guaranteed to retrieve the most recent version of the file available.

Xata provides three ways to expose a file's URL to any request. This allows you to build a range of products from public facing websites, to more security-minded applications that need to think through authentication. Files are secure by default, with action required through these methods to provide access.

Authenticated URLs are private URLs that can be used to access a file with a valid Xata API key. They are available in the url field of a file object if the file is not configured for public access.

These URLs are especially useful if you need quick, high-throughput, and concurrent download access, with cached data.

// Disallow this User photo to be accessed publicly at any time by the URL alone
const user = await xata.db.Users.update('record_id', {
  photo: {
    enablePublicUrl: false
  }
});
 
// Retrieve the private URL on the file in the "photo" column
const { url } = user.photo.transform({ quality: 50 });

# Disallow this User photo to be accessed publicly at any time by the URL alone
user = xata.records().update("Users", "record_id", {
  "photo": {
    "enablePublicUrl": False
  }
})
 
# Retrieve the private URL on the file in the "photo" column
url = xata.files().transform_url("<photo.signedUrl>", {"quality": 50})

recordsClient, _ := xata.NewRecordsClient()
user, _ := recordsClient.Update(context.TODO(), xata.UpdateRecordRequest{
  RecordRequest: xata.RecordRequest{
    TableName: "Users",
  },
  RecordID: "record_id",
  Body: map[string]*xata.DataInputRecordValue{
    "photo": xata.ValueFromInputFile(xata.InputFile{
      EnablePublicUrl: false,
    }),
  },
})
 
// Transform API not implemented yet

A Xata file can be configured for public access, resulting in a URL that is publicly available without the need of an API key or a signature. A public URL does not expire and offers access until the file is reconfigured to remove public access. To enable public access for a file, you can set the enablePublicUrl field to true in the file's configuration. This is particularly useful for public websites and for sharing public content.

You can use the public URL directly without writing code. For instance:

https://us-east-1.storage.xata.sh/4u1fh2o6p10blbutjnphcste94 is available publicly.

// Allow this user's photo to be accessed publicly at any time by the URL alone
const user = await xata.db.Users.update('record_id', {
  photo: {
    enablePublicUrl: true
  }
});
 
// Retrieve the public URL on the file in the "photo" column
const { url } = user.photo.transform({ quality: 50 });

# Allow this user's photo to be accessed publicly at any time by the URL alone
user = xata.records().update("Users", "record_id", {
  "photo": {
    "enablePublicUrl": True
  }
})
 
# Retrieve the public URL on the file in the "photo" column
url = xata.files().transform("https://us-east-1.storage.xata.sh/4u1fh2o6p10blbutjnphcste94", {"quality": 50})

recordsClient, _ := xata.NewRecordsClient()
user, _ := recordsClient.Update(context.TODO(), xata.UpdateRecordRequest{
  RecordRequest: xata.RecordRequest{
    TableName: "Users",
  },
  RecordID: "record_id",
  Body: map[string]*xata.DataInputRecordValue{
    "photo": xata.ValueFromInputFile(xata.InputFile{
      EnablePublicUrl: true,
    }),
  },
})
 
// Transform API not implemented yet

Within the Xata UI you can make files public by default for the column by setting Make files public by default when creating the column. This can also be done programtically by setting a parameter on the file column in the schema.

A signed URL offers authenticated access to a file without requiring an API key. The URL contains the key (signature) within it, so anyone holding the URL can get access to the file. Because of their on demand nature, signed URLs need to be requested directly and do not come along with the default request on the record.

// Returns the signed URL for records[0].photo.signedUrl
const records = await xata.db.Users.select(['id', 'name', 'photo.*', 'photo.signedUrl']).getMany();
 
// Returns an empty string for records[0].photo.signedUrl
const records = await xata.db.Users.getMany();

# Returns the signed URL for records[0]["photo"]["signedUrl"]
records = xata.data().query("Users", {
  "columns": ["id", "name", "photo.*", "photo.signedUrl"]
})
 
# Returns an empty string for records[0]["photo"]["signedUrl"]
records = xata.data().query("Users")

client, _ := xata.NewSearchAndFilterClient()
 
// Returns the signed URL for records[0]["photo"]["signedUrl"]
records, _ := client.Query(context.TODO(), xata.QueryTableRequest{
  TableName: "Users",
  Payload: xata.QueryTableRequestPayload{
    Columns: []string{"id", "name", "photo.*", "photo.signedUrl"},
  },
})
 
// Returns an empty string for records[0]["photo"]["signedUrl"]
records, _ := client.Query(context.TODO(), xata.QueryTableRequest{
  TableName: "Users",
  Payload: xata.QueryTableRequestPayload{},
})

Signed URLs have a configurable time to live (TTL), which specifies when access to the URL expires. To avoid permanent public access, the TTL can be set to a maximum of 24h. You can modify the timeout duration of the signed URL for each file by adjusting the signedUrlTimeout field within the file object. Please note that signedUrlTimeout expects a positive number, defining the timeout duration in seconds. The default value is set to 60 seconds.

Use signed URLs when you need temporary access without revealing the API key, such as rendering an image without disclosing a permanent image URL.

// Set the timeout to 10 minutes for this user's photo
const user = await xata.db.Users.update('record_id', {
  photo: {
    signedUrlTimeout: 600 // In seconds
  }
});
 
// Retrieve the signed URL on the file in the "photo" column
const { signedUrl } = user.photo.transform({ quality: 50 });

# Set the timeout to 10 minutes for this user's photo
user = xata.records().update("Users", "record_id", {
  "photo": {
    "signedUrlTimeout": 600 # In seconds
  }
})
 
# Get the transformed image
img = xata.files().transform("https://us-east-1.storage.xata.sh/4u1fh2o6p10blbutjnphcste94", {
  "quality": 50
})

An upload URL provides a secure and authenticated method to upload or update a file without requiring an API key. This URL includes a time-limited signature, that ensures that only users with the URL can upload a file within a set time frame.

The URLs are similar to signed URLs in that they must be specifically requested and are not automatically included with a standard record request. This improves security by controlling access and limiting the window during which uploads can occur.

HTTP PUT is the only valid method for a request using the upload URL.

An upload URL is assigned to a specific file. Subsequent uploads using the same URL will overwrite the existing file content, instead of generating additional files. Every time the URL is used, it updates the same specific file it's linked to.

// Returns the upload URL for records[0].photo.uploadUrl
const records = await xata.db.Users.select(['id', 'name', 'photo.*', 'photo.uploadUrl']).getMany();
 
// Returns an empty string for records[0].photo.uploadUrl
const records = await xata.db.Users.getMany();

# Returns the signed URL for records[0]["photo"]["uploadUrl"]
records = xata.data().query("Users", {
  "columns": ["id", "name", "photo.*", "photo.uploadUrl"]
})
 
# Returns an empty string for records[0]["photo"]["uploadUrl"]
records = xata.data().query("Users")

client, _ := xata.NewSearchAndFilterClient()
 
// Returns the signed URL for records[0]["photo"]["uploadUrl"]
records, _ := client.Query(context.TODO(), xata.QueryTableRequest{
  TableName: "Users",
  Payload: xata.QueryTableRequestPayload{
    Columns: []string{"id", "name", "photo.*", "photo.uploadUrl"},
  },
})
 
// Returns an empty string for records[0]["photo"]["uploadUrl"]
records, _ := client.Query(context.TODO(), xata.QueryTableRequest{
  TableName: "Users",
  Payload: xata.QueryTableRequestPayload{},
})

Upload URLs have a configurable time to live (TTL), which specifies when access to the URL expires. The default is 24h. You can modify the timeout duration of the upload URL for each file by adjusting the uploadUrlTimeout field within the file object. The value represents a duration in seconds.

Use upload URLs when you need temporary write access without revealing the API key, such as exposing uploads in a web application.

// Set the timeout to 10 minutes for the photo upload URL
const user = await xata.db.Users.update('record_id', {
  photo: {
    uploadUrlTimeout: 600 // Time in seconds
  }
});

# Set the timeout to 10 minutes for the photo upload URL
user = xata.records().update("Users", "record_id", {
  "photo": {
    "uploadUrlTimeout": 600 # Time in seconds
  }
})

// Set the timeout to 10 minutes for the photo upload URL
recordsClient, _ := xata.NewRecordsClient()
user, _ := recordsClient.Update(context.TODO(), xata.UpdateRecordRequest{
  RecordRequest: xata.RecordRequest{
    TableName: "Users",
  },
  RecordID: "record_id",
  Body: map[string]*xata.DataInputRecordValue{
    "photo": xata.ValueFromInputFile(xata.InputFile{
      UploadUrlTimeout: xata.Int(600), // Time in seconds
    }),
  },
})

The SDK documentation for file attachments contains a more detailed pattern for using an uploadUrl in a typical web scenario. The sample gallery app repo on Github also documents a complete example of the pattern for Next.js.