Database
Mysql Tutorial
Automated CSV and Excel Data Import to MySQL and PostgreSQL Databases Using PHP
Automated CSV and Excel Data Import to MySQL and PostgreSQL Databases Using PHP
Nov 12, 2024 pm 05:59 PM
To automate data transfer from a CSV or Excel file to both a MySQL and PostgreSQL database using PHP, follow these steps:
Prerequisites
-
Install necessary libraries:
- PHP's PDO extension for MySQL and PostgreSQL.
- PHPExcel library (or PhpSpreadsheet if available, but we'll use PHPExcel as it's more compatible with PHP 5.6).
Download the PHPExcel library and include it in your project directory.
Step 1: Set Up Database Connections
We'll use PDO to connect to both MySQL and PostgreSQL.
<?php
// MySQL connection
$mysqlHost = 'localhost';
$mysqlDB = 'mysql_database';
$mysqlUser = 'mysql_user';
$mysqlPassword = 'mysql_password';
try {
$mysqlConnection = new PDO("mysql:host=$mysqlHost;dbname=$mysqlDB", $mysqlUser, $mysqlPassword);
$mysqlConnection->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
echo "Connected to MySQL successfully.<br>";
} catch (PDOException $e) {
die("MySQL connection failed: " . $e->getMessage());
}
// PostgreSQL connection
$pgHost = 'localhost';
$pgDB = 'pgsql_database';
$pgUser = 'pgsql_user';
$pgPassword = 'pgsql_password';
try {
$pgConnection = new PDO("pgsql:host=$pgHost;dbname=$pgDB", $pgUser, $pgPassword);
$pgConnection->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
echo "Connected to PostgreSQL successfully.<br>";
} catch (PDOException $e) {
die("PostgreSQL connection failed: " . $e->getMessage());
}
?>
Step 2: Load Data from CSV or Excel File
We will create a function that reads either a CSV or Excel file and returns the data as an array.
<?php
require 'path/to/PHPExcel.php';
function readFileData($filePath) {
$fileType = strtolower(pathinfo($filePath, PATHINFO_EXTENSION));
if ($fileType === 'csv') {
$data = [];
if (($handle = fopen($filePath, 'r')) !== false) {
while (($row = fgetcsv($handle, 1000, ',')) !== false) {
$data[] = $row;
}
fclose($handle);
}
return $data;
} elseif ($fileType === 'xls' || $fileType === 'xlsx') {
$data = [];
$excel = PHPExcel_IOFactory::load($filePath);
$sheet = $excel->getActiveSheet();
foreach ($sheet->getRowIterator() as $row) {
$rowData = [];
$cellIterator = $row->getCellIterator();
$cellIterator->setIterateOnlyExistingCells(false);
foreach ($cellIterator as $cell) {
$rowData[] = $cell->getValue();
}
$data[] = $rowData;
}
return $data;
} else {
throw new Exception("Unsupported file format");
}
}
?>
Step 3: Transfer Data to MySQL and PostgreSQL
Define functions to insert data into both MySQL and PostgreSQL. This example assumes the data is an array of arrays, where each inner array represents a row in the database.
<?php
function insertIntoMySQL($mysqlConnection, $data) {
$query = "INSERT INTO your_mysql_table (column1, column2, column3) VALUES (?, ?, ?)";
$stmt = $mysqlConnection->prepare($query);
foreach ($data as $row) {
$stmt->execute($row);
}
echo "Data inserted into MySQL successfully.<br>";
}
function insertIntoPostgreSQL($pgConnection, $data) {
$query = "INSERT INTO your_pg_table (column1, column2, column3) VALUES (?, ?, ?)";
$stmt = $pgConnection->prepare($query);
foreach ($data as $row) {
$stmt->execute($row);
}
echo "Data inserted into PostgreSQL successfully.<br>";
}
?>
Step 4: Putting It All Together
Load data from the file, then pass it to each function to insert into both MySQL and PostgreSQL.
<?php
$filePath = 'path/to/yourfile.csv'; // or .xls / .xlsx
try {
$data = readFileData($filePath);
insertIntoMySQL($mysqlConnection, $data);
insertIntoPostgreSQL($pgConnection, $data);
} catch (Exception $e) {
echo "Error: " . $e->getMessage();
}
?>
Example Execution
- Make sure the MySQL and PostgreSQL databases and tables (your_mysql_table, your_pg_table) are set up and have the correct columns (column1, column2, column3).
- Place your CSV or Excel file in the specified path ($filePath).
- Run this PHP script from the command line or a browser (if on a web server).
This script will read the data from the specified file and insert it into both databases.
Connect with me:@ LinkedIn and checkout my Portfolio.
Please give my GitHub Projects a star ??
The above is the detailed content of Automated CSV and Excel Data Import to MySQL and PostgreSQL Databases Using PHP. For more information, please follow other related articles on the PHP Chinese website!
Hot AI Tools
Undress AI Tool
Undress images for free
Undresser.AI Undress
AI-powered app for creating realistic nude photos
AI Clothes Remover
Online AI tool for removing clothes from photos.
Clothoff.io
AI clothes remover
Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!
Hot Article
Hot Tools
Notepad++7.3.1
Easy-to-use and free code editor
SublimeText3 Chinese version
Chinese version, very easy to use
Zend Studio 13.0.1
Powerful PHP integrated development environment
Dreamweaver CS6
Visual web development tools
SublimeText3 Mac version
God-level code editing software (SublimeText3)
Handling NULL Values in MySQL Columns and Queries
Jul 05, 2025 am 02:46 AM
When handling NULL values ??in MySQL, please note: 1. When designing the table, the key fields are set to NOTNULL, and optional fields are allowed NULL; 2. ISNULL or ISNOTNULL must be used with = or !=; 3. IFNULL or COALESCE functions can be used to replace the display default values; 4. Be cautious when using NULL values ??directly when inserting or updating, and pay attention to the data source and ORM framework processing methods. NULL represents an unknown value and does not equal any value, including itself. Therefore, be careful when querying, counting, and connecting tables to avoid missing data or logical errors. Rational use of functions and constraints can effectively reduce interference caused by NULL.
Performing logical backups using mysqldump in MySQL
Jul 06, 2025 am 02:55 AM
mysqldump is a common tool for performing logical backups of MySQL databases. It generates SQL files containing CREATE and INSERT statements to rebuild the database. 1. It does not back up the original file, but converts the database structure and content into portable SQL commands; 2. It is suitable for small databases or selective recovery, and is not suitable for fast recovery of TB-level data; 3. Common options include --single-transaction, --databases, --all-databases, --routines, etc.; 4. Use mysql command to import during recovery, and can turn off foreign key checks to improve speed; 5. It is recommended to test backup regularly, use compression, and automatic adjustment.
Calculating Database and Table Sizes in MySQL
Jul 06, 2025 am 02:41 AM
To view the size of the MySQL database and table, you can query the information_schema directly or use the command line tool. 1. Check the entire database size: Execute the SQL statement SELECTtable_schemaAS'Database',SUM(data_length index_length)/1024/1024AS'Size(MB)'FROMinformation_schema.tablesGROUPBYtable_schema; you can get the total size of all databases, or add WHERE conditions to limit the specific database; 2. Check the single table size: use SELECTta
Handling character sets and collations issues in MySQL
Jul 08, 2025 am 02:51 AM
Character set and sorting rules issues are common when cross-platform migration or multi-person development, resulting in garbled code or inconsistent query. There are three core solutions: First, check and unify the character set of database, table, and fields to utf8mb4, view through SHOWCREATEDATABASE/TABLE, and modify it with ALTER statement; second, specify the utf8mb4 character set when the client connects, and set it in connection parameters or execute SETNAMES; third, select the sorting rules reasonably, and recommend using utf8mb4_unicode_ci to ensure the accuracy of comparison and sorting, and specify or modify it through ALTER when building the library and table.
Aggregating data with GROUP BY and HAVING clauses in MySQL
Jul 05, 2025 am 02:42 AM
GROUPBY is used to group data by field and perform aggregation operations, and HAVING is used to filter the results after grouping. For example, using GROUPBYcustomer_id can calculate the total consumption amount of each customer; using HAVING can filter out customers with a total consumption of more than 1,000. The non-aggregated fields after SELECT must appear in GROUPBY, and HAVING can be conditionally filtered using an alias or original expressions. Common techniques include counting the number of each group, grouping multiple fields, and filtering with multiple conditions.
Implementing Transactions and Understanding ACID Properties in MySQL
Jul 08, 2025 am 02:50 AM
MySQL supports transaction processing, and uses the InnoDB storage engine to ensure data consistency and integrity. 1. Transactions are a set of SQL operations, either all succeed or all fail to roll back; 2. ACID attributes include atomicity, consistency, isolation and persistence; 3. The statements that manually control transactions are STARTTRANSACTION, COMMIT and ROLLBACK; 4. The four isolation levels include read not committed, read submitted, repeatable read and serialization; 5. Use transactions correctly to avoid long-term operation, turn off automatic commits, and reasonably handle locks and exceptions. Through these mechanisms, MySQL can achieve high reliability and concurrent control.
Connecting to MySQL Database Using the Command Line Client
Jul 07, 2025 am 01:50 AM
The most direct way to connect to MySQL database is to use the command line client. First enter the mysql-u username -p and enter the password correctly to enter the interactive interface; if you connect to the remote database, you need to add the -h parameter to specify the host address. Secondly, you can directly switch to a specific database or execute SQL files when logging in, such as mysql-u username-p database name or mysql-u username-p database name
Managing Character Sets and Collations in MySQL
Jul 07, 2025 am 01:41 AM
The setting of character sets and collation rules in MySQL is crucial, affecting data storage, query efficiency and consistency. First, the character set determines the storable character range, such as utf8mb4 supports Chinese and emojis; the sorting rules control the character comparison method, such as utf8mb4_unicode_ci is case-sensitive, and utf8mb4_bin is binary comparison. Secondly, the character set can be set at multiple levels of server, database, table, and column. It is recommended to use utf8mb4 and utf8mb4_unicode_ci in a unified manner to avoid conflicts. Furthermore, the garbled code problem is often caused by inconsistent character sets of connections, storage or program terminals, and needs to be checked layer by layer and set uniformly. In addition, character sets should be specified when exporting and importing to prevent conversion errors
