Which command can you use to create a new table using data in an existing table?
This document describes how to create and use standard (built-in) tables in BigQuery. For information about creating other table types, see: Show
After creating a table, you can:
For more information about managing tables including updating table properties, copying a table, and deleting a table, see Managing tables. Before you beginBefore creating a table in BigQuery, first:
Table namingWhen you create a table in BigQuery, the table name must be unique per dataset. The table name can:
The following are all examples of valid table names: 5, 6, 7, 8.Caveats:
Create tablesYou can create a table in BigQuery in the following ways:
Required permissionsTo create a table, you need the following IAM permissions:
Additionally, you might require the 9 permission to access the data that you write to the table.Each of the following predefined IAM roles includes the permissions that you need in order to create a table:
Additionally, if you have the 8 permission, you can create and update tables in the datasets that you create.For more information on IAM roles and permissions in BigQuery, see Predefined roles and permissions. Create an empty table with a schema definitionYou can create an empty table with a schema definition in the following ways:
For more information about specifying a table schema, see Specifying a schema. After the table is created, you can load data into it or populate it by writing query results to it. To create an empty table with a schema definition: In the Google Cloud console, go to the BigQuery page. Go to BigQuery The following example creates a table named
For more information about how to run queries, see Running interactive queries. bqUse the 3 command with the 5 or 6 flag. You can supply table schema information inline or via a JSON schema file. Optional parameters include:
9, CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY00, CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY01, CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY02, and CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY03 are not demonstrated here. Refer to the following links for more information on these optional parameters:
If you are creating a table in a project other than your default project, add the project ID to the dataset in the following format: CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY15. To create an empty table in an existing dataset with a schema definition, enter the following: bq mk \ --table \ --expiration integer \ --description description \ --label key_1:value_1 \ --label key_2:value_2 \ project_id:dataset.table \ schema Replace the following:
When you specify the schema on the command line, you cannot include a CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY16 ( CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY17) type, you cannot include a column description, and you cannot specify the column's mode. All modes default to CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY18. To include descriptions, modes, and CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY16 types, supply a JSON schema file instead. Examples: Enter the following command to create a table using an inline schema definition. This command creates a table named CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY20 in CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY21 in your default project. The table expiration is set to 3600 seconds (1 hour), the description is set to CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY22, and the label is set to CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY23. The command uses the 6 shortcut instead of 5. The schema is specified inline as: CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY26. bq mk \ -t \ --expiration 3600 \ --description "This is my table" \ --label organization:development \ mydataset.mytable \ qtr:STRING,sales:FLOAT,year:STRING Enter the following command to create a table using a JSON schema file. This command creates a table named CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY20 in CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY21 in your default project. The table expiration is set to 3600 seconds (1 hour), the description is set to CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY22, and the label is set to CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY23. The path to the schema file is CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY31. bq mk \ --table \ --expiration 3600 \ --description "This is my table" \ --label organization:development \ mydataset.mytable \ /tmp/myschema.json Enter the following command to create a table using an JSON schema file. This command creates a table named CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY20 in CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY21 in CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY34. The table expiration is set to 3600 seconds (1 hour), the description is set to CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY22, and the label is set to CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY23. The path to the schema file is CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY31. bq mk \ --table \ --expiration 3600 \ --description "This is my table" \ --label organization:development \ myotherproject:mydataset.mytable \ /tmp/myschema.json After the table is created, you can update the table's expiration, description, and labels. You can also modify the schema definition. APICall the 4 method with a defined table resource.C#Before trying this sample, follow the C# setup instructions in the BigQuery quickstart using client libraries. For more information, see the BigQuery C# API reference documentation. View on GitHub Feedback
GoBefore trying this sample, follow the Go setup instructions in the BigQuery quickstart using client libraries. For more information, see the BigQuery Go API reference documentation. View on GitHub Feedback
JavaBefore trying this sample, follow the Java setup instructions in the BigQuery quickstart using client libraries. For more information, see the BigQuery Java API reference documentation. View on GitHub Feedback
Node.jsBefore trying this sample, follow the Node.js setup instructions in the BigQuery quickstart using client libraries. For more information, see the BigQuery Node.js API reference documentation. View on GitHub Feedback
PHPBefore trying this sample, follow the PHP setup instructions in the BigQuery quickstart using client libraries. For more information, see the BigQuery PHP API reference documentation. View on GitHub Feedback CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY0 PythonBefore trying this sample, follow the Python setup instructions in the BigQuery quickstart using client libraries. For more information, see the BigQuery Python API reference documentation. View on GitHub Feedback CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY1 RubyBefore trying this sample, follow the Ruby setup instructions in the BigQuery quickstart using client libraries. For more information, see the BigQuery Ruby API reference documentation. View on GitHub Feedback CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY2 Create an empty table without a schema definitionBefore trying this sample, follow the Java setup instructions in the BigQuery quickstart using client libraries. For more information, see the BigQuery Java API reference documentation. View on GitHub Feedback Create a table from a query resultTo create a table from a query result, write the results to a destination table. Open the BigQuery page in the Google Cloud console. Go to the BigQuery page In the Explorer panel, expand your project and select a dataset. Enter a valid SQL query. Click More and then select Query settings. Select the Set a destination table for query results option. In the Destination section, select the Dataset in which you want to create the table, and then choose a Table Id. In the Destination table write preference section, choose one of the following:
Optional: For Data location, choose your location. To update the query settings, click Save. Click Run. This creates a query job that writes the query results to the table you specified. Alternatively, if you forget to specify a destination table before running your query, you can copy the cached results table to a permanent table by clicking the Save Results button above the editor. SQLThe following example uses the 5 statement to create the CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY40 table from data in the public CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY41 table:
For more information about how to run queries, see Running interactive queries. For more information, see Creating a new table from an existing table. bqEnter the CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY42 command and specify the CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY43 flag to create a permanent table based on the query results. Specify the CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY44 flag to use standard SQL syntax. To write the query results to a table that is not in your default project, add the project ID to the dataset name in the following format: CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY15. Optional: Supply the CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY46 flag and set the value to your location. To control the write disposition for an existing destination table, specify one of the following optional flags:
CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY5 Replace the following:
If no write disposition flag is specified, the default behavior is to write the results to the table only if it is empty. If the table exists and it is not empty, the following error is returned: `BigQuery error in query operation: Error processing job CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY57. Examples: Note: These examples query a US-based public dataset. Because the public dataset is stored in the US multi-region location, the dataset that contains your destination table must also be in the US. You cannot query a dataset in one location and write the results to a destination table in another location.Enter the following command to write query results to a destination table named CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY20 in CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY21. The dataset is in your default project. Since no write disposition flag is specified in the command, the table must be new or empty. Otherwise, an CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY60 error is returned. The query retrieves data from the USA Name Data public dataset. CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY6 Enter the following command to use query results to overwrite a destination table named CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY20 in CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY21. The dataset is in your default project. The command uses the CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY48 flag to overwrite the destination table. CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY7 Enter the following command to append query results to a destination table named CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY20 in CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY21. The dataset is in CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY66, not your default project. The command uses the CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY47 flag to append the query results to the destination table. CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY8 The output for each of these examples looks like the following. For readability, some output is truncated. CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY9 APITo save query results to a permanent table, call the CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY68 method, configure a CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY56 job, and include a value for the CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY70 property. To control the write disposition for an existing destination table, configure the CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY71 property. To control the processing location for the query job, specify the CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY49 property in the CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY73 section of the job resource. GoBefore trying this sample, follow the Go setup instructions in the BigQuery quickstart using client libraries. For more information, see the BigQuery Go API reference documentation. bigquery/snippets/querying/bigquery_query_destination_table.go View on GitHub Feedback bq mk \ --table \ --expiration integer \ --description description \ --label key_1:value_1 \ --label key_2:value_2 \ project_id:dataset.table \ schema0 JavaBefore trying this sample, follow the Java setup instructions in the BigQuery quickstart using client libraries. For more information, see the BigQuery Java API reference documentation. To save query results to a permanent table, set the destination table to the desired TableId in a QueryJobConfiguration. View on GitHub bq mk \ --table \ --expiration integer \ --description description \ --label key_1:value_1 \ --label key_2:value_2 \ project_id:dataset.table \ schema1 Node.jsBefore trying this sample, follow the Node.js setup instructions in the BigQuery quickstart using client libraries. For more information, see the BigQuery Node.js API reference documentation. View on GitHub Feedback bq mk \ --table \ --expiration integer \ --description description \ --label key_1:value_1 \ --label key_2:value_2 \ project_id:dataset.table \ schema2 PythonBefore trying this sample, follow the Python setup instructions in the BigQuery quickstart using client libraries. For more information, see the BigQuery Python API reference documentation. To save query results to a permanent table, create a QueryJobConfig and set the destination to the desired TableReference. Pass the job configuration to the query method.View on GitHub Feedback bq mk \ --table \ --expiration integer \ --description description \ --label key_1:value_1 \ --label key_2:value_2 \ project_id:dataset.table \ schema3 Create a table that references an external data sourceAn external data source is a data source that you can query directly from BigQuery, even though the data is not stored in BigQuery storage. For example, you might have data in a different Google Cloud database, in files in Cloud Storage, or in a different cloud product altogether that you would like to analyze in BigQuery, but that you aren't prepared to migrate. For more information, see Introduction to external data sources. Create a table when you load dataWhen you load data into BigQuery, you can load data into a new table or partition, you can append data to an existing table or partition, or you can overwrite a table or partition. You do not need to create an empty table before loading data into it. You can create the new table and load your data at the same time. When you load data into BigQuery, you can supply the table or partition schema, or for supported data formats, you can use schema auto-detection. For more information about loading data, see Introduction to loading data into BigQuery. Control access to tablesTo configure access to tables and views, you can grant an IAM role to an entity at the following levels, listed in order of range of resources allowed (largest to smallest):
You can also restrict data access within tables, by using the following methods:
Access with any resource protected by IAM is additive. For example, if an entity does not have access at the high level such as a project, you could grant the entity access at the dataset level, and then the entity will have access to the tables and views in the dataset. Similarly, if the entity does not have access at the high level or the dataset level, you could grant the entity access at the table or view level. Granting IAM roles at a higher level in the Google Cloud resource hierarchy such as the project, folder, or organization level gives the entity access to a broad set of resources. For example, granting a role to an entity at the project level gives that entity permissions that apply to all datasets throughout the project. Granting a role at the dataset level specifies the operations an entity is allowed to perform on tables and views in that specific dataset, even if the entity does not have access at a higher level. For information on configuring dataset-level access controls, see Controlling access to datasets. Granting a role at the table or view level specifies the operations an entity is allowed to perform on specific tables and views, even if the entity does not have access at a higher level. For information on configuring table-level access controls, see Controlling access to tables and views. You can also create IAM custom roles. If you create a custom role, the permissions you grant depend on the specific operations you want the entity to be able to perform. You can't set a "deny" permission on any resource protected by IAM. For more information about roles and permissions, see Understanding roles in the IAM documentation and the BigQuery IAM roles and permissions. Get information about tablesYou can get information or metadata about tables in the following ways:
Required permissionsAt a minimum, to get information about tables, you must be granted CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY78 permissions. The following predefined IAM roles include CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY78 permissions:
In addition, if a user has 8 permissions, when that user creates a dataset, they are granted CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY82 access to it. CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY82 access gives the user the ability to retrieve table metadata. For more information on IAM roles and permissions in BigQuery, see Access control. Get table informationTo get information about tables: In the navigation panel, in the Resources section, expand your project, and then select a dataset. Click the dataset name to expand it. The tables and views in the dataset appear. Click the table name. In the Details panel, click Details to display the table's description and table information. Optionally, switch to the Schema tab to view the table's schema definition. Issue the If you are getting information about a table in a project other than your default project, add the project ID to the dataset in the following format: CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY15. bq mk \ --table \ --expiration integer \ --description description \ --label key_1:value_1 \ --label key_2:value_2 \ project_id:dataset.table \ schema4 Where:
Examples: Enter the following command to display all information about CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY20 in CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY21. CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY21 is in your default project. bq mk \ --table \ --expiration integer \ --description description \ --label key_1:value_1 \ --label key_2:value_2 \ project_id:dataset.table \ schema5 Enter the following command to display all information about CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY20 in CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY21. CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY21 is in CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY34, not your default project. bq mk \ --table \ --expiration integer \ --description description \ --label key_1:value_1 \ --label key_2:value_2 \ project_id:dataset.table \ schema6 Enter the following command to display only schema information about CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY20 in CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY21. CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY21 is in CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY34, not your default project. bq mk \ --table \ --expiration integer \ --description description \ --label key_1:value_1 \ --label key_2:value_2 \ project_id:dataset.table \ schema7 APICall the CREATE TABLE mydataset.newtable ( x INT64 OPTIONS (description = 'An optional INTEGER field'), y STRUCT < a ARRAY76 method and provide any relevant parameters. GoBefore trying this sample, follow the Go setup instructions in the BigQuery quickstart using client libraries. For more information, see the BigQuery Go API reference documentation. View on GitHub Feedback bq mk \ --table \ --expiration integer \ --description description \ --label key_1:value_1 \ --label key_2:value_2 \ project_id:dataset.table \ schema8 JavaBefore trying this sample, follow the Java setup instructions in the BigQuery quickstart using client libraries. For more information, see the BigQuery Java API reference documentation. View on GitHub Feedback bq mk \ --table \ --expiration integer \ --description description \ --label key_1:value_1 \ --label key_2:value_2 \ project_id:dataset.table \ schema9 Node.jsBefore trying this sample, follow the Node.js setup instructions in the BigQuery quickstart using client libraries. For more information, see the BigQuery Node.js API reference documentation. View on GitHub Feedback bq mk \ -t \ --expiration 3600 \ --description "This is my table" \ --label organization:development \ mydataset.mytable \ qtr:STRING,sales:FLOAT,year:STRING0 PHPBefore trying this sample, follow the PHP setup instructions in the BigQuery quickstart using client libraries. For more information, see the BigQuery PHP API reference documentation. View on GitHub Feedback bq mk \ -t \ --expiration 3600 \ --description "This is my table" \ --label organization:development \ mydataset.mytable \ qtr:STRING,sales:FLOAT,year:STRING1 PythonBefore trying this sample, follow the Python setup instructions in the BigQuery quickstart using client libraries. For more information, see the BigQuery Python API reference documentation. View on GitHub Feedback bq mk \ -t \ --expiration 3600 \ --description "This is my table" \ --label organization:development \ mydataset.mytable \ qtr:STRING,sales:FLOAT,year:STRING2 Get table information using CREATE TABLE mydataset.newtable (
x INT64 OPTIONS (description = 'An optional INTEGER field'),
y STRUCT <
a ARRAY |