Convert XML file into BigQuery table

There is probably a time when your manager, lead, or any stakeholder wants you to load the XML data into a big query table. Then you might wonder why anyone would store queriable data in an XML file. There are many reasons why people do that one possible reason would be, XML frequently complements HTML. While HTML is frequently used to format and display data, XML is frequently utilized to store or transfer data in HTML applications. and data engineers have to create a job on transferring the XML data into a database table for further use.

This article will provide step-by-step instructions to load XML data into the Bigquery table.

For simplicity i am using Cloud shell (google cloud terminal) you can also use local setup for local files. I am using Pandas libraries to read the data from XML and convert it dataframe so that it will be easier you transform the data to columar format. And then use the Pandas API (Applicaiton Programming Interface) to save the data into the Biquery table.

Now that you have created new project and enabled the bigquery api we need to create service account which helps our python script to authenticate and authorise to access data using the Google API.

Create an xml file , i have create a small employee salary xml record rest for this article you can use your actual XML file that you need to use. And upload the File to google cloud shell like we did in the earlier step.

Python pandas is very beautiful library that helps read any type of possible data into dataframes you can install these packages as follows.

Click on the Table name and then click on Preview

Hurray !!!!!! you have loaded your xml data into bigquery table. there are lots of possibilities out there to this kind of transformation. keep learing and keep growing.

Related posts:

Pekhill Organic Boswellia(Boswellia Serrata) Extract is a standardized product and contains 65% Boswellia acid (Titration Method). We always provide our customers with pesonlized solutions. If you…

People sometimes ask me what my approach to writing is, how exactly do I go about it? I wish I had a pat answer I really do. The problem is I constantly change from day to day, from hour to hour. I…

La pandemia ha generado nuevas necesidades de los ciudadanos. Para atenderlas, la Secretaría de Gobierno Digital (Gob.pe) ha creado 7 servicios digitales.