Skip to content

Changelog

2.2.1 - 2026-03-25

Authors: Nicole Tometich

Changes:

  • Minor adjustment on PgSqlHandler to fix error when inserting JSON type columns to postgres

2.2.0 - 2026-03-24

Authors: Bruno Macedo

Changes:

  • Added abstract table_exists(schema, table_name) method to the SqlHandler base class. Each handler implements it using the most efficient database-native query available:
    • PgSqlHandler: queries pg_class / pg_namespace system catalogs — covers tables, views, and materialized views.
    • SqlLiteHandler: queries sqlite_master — covers tables and views.
    • MsSqlHandler: queries INFORMATION_SCHEMA.TABLES — defaults schema to "dbo" when None is passed.
    • MySqlHandler: raises NotImplementedError (not yet implemented).
  • Internal callers of get_table_names() in SqlHandler, PgSqlHandler, and SqlLiteHandler were refactored to use table_exists() instead. This avoids fetching the full list of table names for existence checks, improving performance in schemas with many tables. Affected methods: _prepare_df_for_sql(), check_df_columns(), create_table(), delete_table(), and get_table_pkey().
  • check_df_columns() in SqlHandler now gracefully handles custom/user-defined database types (e.g. PostgreSQL ENUM types). Columns whose database type is not in the handler's known type mapping are skipped instead of raising a ValueError, allowing upserts into tables that use application-defined types.

2.1.0 - 2026-03-24

Authors: Bruno Macedo

Changes:

  • Added auto_close_connection parameter to SqlConnProperties to allow for automatic connection closure after each query, improving resource management for databases. By default, this is set to True, meaning the connection will be closed automatically after each query. Setting it to False allows for manual control of the connection lifecycle, which can be useful for long-running operations or when using transactions. When enabled, this feature ensures that connections are not left open unintentionally, reducing the risk of database locks and improving overall performance when executing multiple queries in succession.
  • Other improvements and bug fixes.

2.0.3 - 2026-03-24

Authors: Bruno Macedo

Changes:

  • Minor bug fixes.

2.0.2 - 2026-03-23

Authors: Bruno Macedo

Changes:

  • Disabled time zone conversion for ADBC in all handlers except PostgreSQL.

2.0.1 - 2026-03-23

Authors: Bruno Macedo

Changes:

  • Minor bug fixes to support SQLite not allowing time zone.
  • Removed version requirements for polars and psycopg in pyproject.toml.

2.0.0 - 2026-03-23

Authors: Bruno Macedo

Breaking Changes:

  • SQL connections are now lazy by default (skip_connect=True). The connection is only established when the first query is executed or when reconnect() is called manually. Existing code that relies on the connection being established at construction time should pass skip_connect=False explicitly.
  • Changed default connection order in MsSqlHandler to try mssql-python first, then pyodbc, then pymssql. Set driver="pymssql" to force the old pymssql-first behavior.
  • read_to_polars() now strips time zone information from datetime columns by default (remove_time_zone=True), matching the existing read_to_pandas() behavior.

New Features:

  • Added ADBC (Apache Arrow Database Connectivity) support for read_to_pandas() and read_to_polars() across PgSqlHandler, MsSqlHandler, MySqlHandler, and SqlLiteHandler. ADBC uses Apache Arrow's columnar format natively, providing significantly faster reads for large datasets. Enabled by default via the use_adbc parameter.
  • ADBC connections are cached across reads for server-based databases, reducing connection overhead. For SqlLiteHandler, caching is disabled to avoid stale snapshot reads after writes.
  • ADBC timestamps (returned as UTC) are automatically converted to the session time zone configured in the connection properties, ensuring consistent behavior with the native driver path.
  • Added remove_time_zone parameter to read_to_polars() to allow stripping time zone information from datetime columns, matching the existing read_to_pandas() functionality.
  • Added _strip_quotes() utility to ConnHandler for handling connection property values that may contain embedded double quotes from database-stored configurations.

Dependencies:

  • Added adbc-driver-manager, adbc-driver-postgresql, adbc-driver-sqlite, and mssql-python as project dependencies.
  • Suppressed Polars arrow.opaque extension type warning by setting POLARS_UNKNOWN_EXTENSION_TYPE_BEHAVIOR=load_as_storage at import time.

1.12.5 - 2026-03-16

Authors: Bruno Macedo

Changes:

  • Changed argument order in SqlHandler.execute() method to avoid errors when passing positional arguments to the method.

1.12.5 - 2026-03-15

Authors: Bruno Macedo

Changes:

  • execute() method in SqlHandler now returns the result of the query execution.
  • Added rowcount property to SqlHandler to allow retrieving the number of rows affected by the last query executed. This is useful for queries that do not return a result set, such as INSERT, UPDATE and DELETE queries.

1.12.4 - 2026-02-19

Authors: Bruno Macedo

Changes:

  • Removed accept-encoding header from default headers in HttpConnProperties to avoid errors when connecting to some servers that do not support compressed responses.

1.12.3 - 2026-02-18

Authors: Bruno Macedo

Changes:

  • Added custom headers to HttpConnProperties to avoid being blocked by WAFs and bot-protection systems when connecting to some servers. By default, the headers include a User-Agent string with the package name and version, as well as common headers for content type, accepted languages, and encoding.
  • Added CceeHttpHandler to connect to CCEE API.

1.12.2 - 2026-01-29

Authors: Bruno Macedo

Changes:

  • Fixed FtpHandler.get_file not closing the file after reading it when using a file-like object as destination.

1.12.1 - 2026-01-28

Authors: Bruno Macedo

Changes:

  • Added new argument max_conn_retries to all handlers to split the connection retry limit and the query retry limit. This allows to have a lower number of connection retries while keeping a higher number of query retries.

1.12.0 - 2026-01-27

Authors: Bruno Macedo

Changes:

  • Changed retry logic in all connection handlers to use exponential backoff strategy with tenacity library. Now the number of retries, maximum wait time, minimum wait time and multiplier can be configured in the connection handler constructor.

1.11.2 - 2026-01-19

Authors: Bruno Macedo

Changes:

  • Added ignore_null_cols parameter to polars_to_sql and pandas_to_sql methods in SQL handlers to allow ignoring columns with only null values when writing to the database.
  • Added BIGINT segmentation in sql data types for both pandas and polars in PgSqlHandler.

1.11.1 - 2026-01-11

Authors: Bruno Macedo

Changes:

  • polars_to_sql now supports inserting or updating columns with array data types in PgSqlHandler.

1.11.0 - 2026-01-11

Authors: Bruno Macedo

Changes:

  • Added update_only parameter to polars_to_sql and pandas_to_sql methods in SQL handlers to allow updating existing records without inserting new ones.

1.10.0 - 2026-01-09

Authors: Bruno Macedo

Changes:

  • Added return_cols parameter to polars_to_sql and pandas_to_sql methods in SQL handlers to allow returning specific columns after an upsert operation.
  • Added polars_to_sql method to SqlLiteHandler.

1.9.0 - 2025-12-22

Authors: Bruno Macedo

Changes:

  • Upgraded to Python 3.13.

1.8.2 - 2025-12-15

Authors: Bruno Macedo

Changes:

  • Made infer_schema_length=None the default in read_to_polars method of SQL handlers to avoid errors while trying to infer schema with small datasets.

1.8.1 - 2025-11-27

Authors: Nicole Oska

Changes:

  • Added the MySqlHandler on documentation.

1.8.0 - 2025-11-25

Authors: Nicole Oska

Changes:

  • Added the MySqlHandler class and tests.

1.7.2 - 2025-11-24

Authors: Bruno Macedo

Changes:

  • Added option to set pg_settings in PgSqlHandler to set additional PostgreSQL settings at the start of the connection.

1.7.1 - 2025-09-10

Authors: Nicole Tometich

Changes:

  • Minor change to avoid "AttributeError: 'psycopg_c.pq.PGconn' object has no attribute 'used_gssapi'" on psycopg 3.2.10 version

1.7.0 - 2025-08-13

Authors: Bruno Macedo

Changes:

  • Changed HTTP classes to use httpx instead of requests module.
  • Added OnsHttpHandler to connect to SAGER and SINAPSE.
  • Reorganized tests.

1.6.1 - 2025-07-12

Authors: Bruno Macedo

Changes:

  • Changed build backend to uv_build for better packaging and deployment.

1.6.0 - 2025-07-09

Authors: Bruno Macedo

Changes:

  • Added polars_to_sql method to PgSqlHandler.

1.5.3 - 2025-06-30

Authors: Bruno Macedo

Changes:

  • Fixed type hints for skip_row_check.

1.5.2 - 2025-06-24

Authors: Bruno Macedo

Changes:

  • Added compatibility for Path object in FtpHandler.

1.5.1 - 2025-06-23

Authors: Bruno Macedo

Changes:

  • Added option to allow empty strings (endpoint) in http requests.

1.5.0 - 2025-06-19

Authors: Bruno Macedo

Changes:

  • Added read_to_polars to SQL classes.
  • Added pydantic validate call to all methods.
  • Added ruff.toml file to configure ruff.

1.4.1 - 2025-04-28

Authors: Bruno Macedo

Changes:

  • Added headers argument to HttpConnProperties to allow for custom headers to be set in the session.
  • HttpHandler now allows for requests to be made to empty endpoints.

1.4.0 - 2025-04-10

Authors: Bruno Macedo

Changes:

  • Added option to specify payload_kwargs to be added to every request in any subclass of HttpHandler. This allows to add something that is fixed for every request, like api_key or credentials.

1.3.10 - 2025-02-18

Authors: Bruno Macedo

Changes:

  • Added option to select pymssql directly in MsSqlHandler to avoid errors when connecting to SCADA SQL servers.

1.3.9 - 2025-02-05

Authors: Bruno Macedo

Changes:

  • Changed API URL for bazefield password login.

1.3.8 - 2024-12-20

Authors: Bruno Macedo

Changes:

  • Added schema argument to SQL connection. In postgres, when this is set, the search path is set to the schema name.

1.3.7 - 2024-12-15

Authors: Bruno Macedo

Changes:

  • Removed unnecessary log messages when connections was successful.

1.3.6 - 2024-12-09

Authors: Bruno Macedo

Changes:

  • Added skip_row_check to if_exists in pandas_to_sql.

1.3.5 - 2024-10-14

Authors: Bruno Macedo

Changes:

  • Removed hard requirement of api key or credentials in HttpConnProperties.
  • Moved _connect from BazeHttpHandler to HttpHandler to allow for more flexibility in the connection process. Only authentication is now done in BazeHttpHandler.

1.3.4 - 2024-09-09

Authors: Bruno Macedo

Changes:

  • Added option to get columns from views and materialized views in PgSqlHandler.get_table_columns().

1.3.3 - 2024-09-02

Authors: Bruno Macedo

Changes:

  • Changed query to upsert eh values in PgSqlHandler.pandas_to_sql to avoid errors when there is a duplicate key in the data being inserted. Also broke the insert process in chunks of 100000 rows to avoid errors.

1.3.2 - 2024-09-02

Authors: Bruno Macedo

Changes:

  • Set SSL default requirement to False in HttpConnProperties to avoid errors when connecting to non-secure servers.

1.3.1 - 2024-08-27

Authors: Bruno Macedo

Changes:

  • Bug fix in upsert of SqlLiteHandler.pandas_to_sql.

1.3.0 - 2024-08-16

Authors: Bruno Macedo

Changes:

  • Major changes in documentation. Now we are using MkDocs to generate the documentation and Material for MkDocs as the theme.

1.2.1 - 2024-07-25

Authors: Bruno Macedo

Changes:

  • Added option to forcefully skip retries in SqlHandler read_to_pandas() and execute() methods, skipping the maximum number of retries set in the ConnProperties object.

1.2.0 - 2024-07-14

Authors: Bruno Macedo

Changes:

  • Added better support for time zone aware in SqlHandler, specially in read_to_pandas() method.

1.1.0 - 2024-07-09

Authors: Bruno Macedo

Changes:

  • Upgraded to python 3.12.

1.0.8 - 2024-06-30

Authors: Bruno Macedo

Changes:

  • In MsSqlHandler fixed TDS version when connecting using pymssql to allow upgrading pymssql to version higher than 2.2.8.

1.0.7 - 2024-05-23

Authors: Bruno Macedo

Changes:

  • Changed FtpHandler._close() to use FTP quit method instead of close method, to avoid errors when closing the connection.

1.0.6 - 2024-05-13

Authors: Bruno Macedo

Changes:

  • Minor bug fix in FtpHandler.delete_file().

1.0.5 - 2024-04-15

Authors: Bruno Macedo

Changes:

  • Changed to new Bazefield URL.

1.0.4 - 2024-04-10

Authors: Bruno Macedo

Changes:

  • Fixed bug in SqlHandler._get_pandas_sql_dtypes().

1.0.3 - 2024-03-10

Authors: Bruno Macedo

Changes:

  • Added option to specify conflict_cols in pandas_to_sql method of SqlHandler subclasses. This allows to specify the columns to be used in the ON CONFLICT clause.

1.0.2 - 2024-03-06

Authors: Bruno Macedo

Changes:

  • Added notices attribute to PgSqlHandler to store the notices from the server. It allows to retrieve any notices from previous queries.
  • pyarrow as default dtype_backend in SqlHandler.read_to_pandas().
  • Added post_convert option to SqlHandler.read_to_pandas() for converting to pyarrow or Numpy nullable after reading that, which solves some issues when reading JSON columns from PostgreSQL.

1.0.1 - 2024-03-04

Authors: Bruno Macedo

Changes:

  • Added skip_connect option to ConnHandler constructor, enabling the creation of a handler without connecting to the server.

1.0.0 - 2024-03-03

Authors: Bruno Macedo

Changes:

  • Major changes to PgSqlHandler to add new functionality such as saving data from a pandas DataFrame to a table in the database.
  • SqlHandler._get_pandas_sql_dtypes() much improved, now checks for the columns data type and only falls to checking column values if could not infer the data type from the column dtype.
  • Added create_table and delete_table methods to PgSqlHandler and SqlLiteHandler.
  • More detailed README file.

0.4.8 - 2024-03-01

Authors: Bruno Macedo

Changes:

  • Moved installation to pyproject.toml.
  • Refactored considering ruff suggestions.

0.4.7 - 2024-02-07

Authors: Bruno Macedo

Changes:

  • Fixed retries in BazeHttpHandler.

0.4.6 - 2024-02-03

Authors: Bruno Macedo

Changes:

  • Adjusted pool_maxsize in BazeHttpHandler to 32 to allow concurrent requests to the Bazefield API.

0.4.5 - 2023-08-10

Authors: Bruno Macedo

Changes:

  • Forced use of pymssql 2.2.8 to avoid errors with newer versions when connecting to SCADA SQL servers.

0.4.4 - 2023-08-08

Authors: Bruno Macedo

Changes:

  • Changed functions in all core connection handlers to show the first error and not the last in case failed after multiple attempts.

0.4.3 - 2023-06-19

  • Added support to pyarrow pandas backend.

0.4.2 - 2023-06-15

  • Added type checks for all ConnProperties subclasses, avoiding errors when instantiating them. As they are data classes this was done using the __post_init__ method.
  • Changed chunk size in to_sql method of SqlLitHandler.pandas_to_sql to be dynamically set based on the input DataFrame.

0.4.1 - 2023-05-22

  • Minor bug fix when using ConnHandler in with statement.
  • Added custom dtype mapping to SqlLiteHandler.

0.4.0 - 2023-05-18

  • Added SqlLiteConnProperties and SqlLiteHandler classes.
  • Added class method _get_pandas_sql_dtypes to SqlHandler to convert Pandas dtypes to SQL dtypes.
  • Added many abstract methods to SqlHandler to be implemented by subclasses. These methods were already implemented in SqlLiteHandler but need to be implemented in MsSqlHandler and PgSqlHandler.

0.3.0 - 2023-05-11

  • Added capabilities to ConnHandler to be used in a with statement.

0.2.7 - 2023-05-07

  • Changed default retry_wait_time to a lower value in all modules.

0.2.6 - 2023-04-22

  • Added option to pass positional and keyword arguments to the execute method of SqlHandler.

0.2.5 - 2023-04-19

  • Added execute method to SqlHandler. This is used to execute queries using the cursor.

0.2.4 - 2023-04-19

  • Changed SqlHandler to do everything in read_to_pandas inside a try/except block to avoid erros if the connection is lost.

0.2.3 - 2023-04-18

  • Added Encrypt=no to the ODBC connection string of sql_ms.py to improve compatibility with older versions of SQL server.

0.2.2 - 2023-02-09

  • Made MsSqlHandler compatible with ODBC driver version 18.

0.2.1 - 2023-02-06

  • Changed FtpHandler to connect using passive mode.

0.2.0 - 2023-01-26

  • Added FtpConnProperties and FtpHandler classes.
  • ConnHandler.reconnect() now attempts to close the connection before connecting again.

0.1.1 - 2023-01-10

  • Added the cursor() method to SqlHandler class.

0.1.0 - 2023-01-09

  • Changed package name to echo_connhandler to allow for a more broader use.
  • Added HttpConnProperties and HttpHandler as abstract classes to handle connection to HTTP servers.
  • Added to BazeHttpHandler to connect to Bazefield.

0.0.2 - 2023-01-04

  • Minor bug fixes.

0.0.1 - 2023-01-04

  • Initial release.