Release Notes Fivetran HVR Version 6.2.0/0 (linux_glibc2.12-x64-64bit) 2024-09-25 COMPATIBILITY FOR PLATFORM LINUX_GLIBC2.12-X64-64BIT ---------------------------------------------------- Fivetran HVR version 6.x is not compatible with the versions 5.x and 4.x. So, ensure that the HVR version is 6.x on the Hub machine and Agent machine. - Operating System: Linux (x86 64 bit) based on GLIBC 2.12 and higher. - Operating System: Red Hat Enterprise Server version 6 and higher - Operating System: SuSE Enterprise Server version 12.0 and higher This release contains 64-bit executables which will not run on a 32-bit Linux machine. - Ingres: 64-bit (a64.lnx) version 10.0, 10.1, 10.2, 11.0, 11.1 and 11.2 (with patch 15820 or later) Capture of X100 tables is not supported - Vector: version 4.x, 5.0, 5.1 and 6.0 as hub and target, not capture. - Vector Hadoop Edition: version 4.x as hub and target, not capture. - Oracle: version 11.1, 11.2, 12.1, 12.2, 18, 19c and 21c, Exadata - Netweaver: version 7.5 and higher - DB2 on Linux, Unix, Windows: version 10.5, 11.1 and 11.5 - DB2 for i: version 7.2, 7.3, 7.4 and 7.5 Requires ODBC driver manager UnixODBC Requires IBM i Access Client Solutions ODBC Driver 64-bit - DB2 for z/OS: version 11.1, 12.1 and 13.1 Version 11.1 requires DB2 client, DB2 server or DB2 Connect version 10.5 or higher Versions 12.1 and 13.1 requires DB2 client, DB2 server or DB2 Connect version 11.1 fixpack 1 or higher - PostgreSQL: version 9.5, 9.6, 10, 11, 12, 13, 14, 15 and 16 Aurora PostgreSQL: version 11, 12, 13, 14, 15 and 16 - MySQL: version 5.6, 5.7 and 8 MariaDB: version 10.0, 10.1, 10.2, 10.3 and 10.4 Aurora MySQL: version 1, 2 and 3. - SQL Server: version 2017 and Azure SQL Database Requires ODBC driver manager UnixODBC Requires ODBC Driver for SQL Server 11 or higher - Snowflake: version 2.x as target, but not hub or capture Requires ODBC driver manager UnixODBC Requires Snowflake ODBC driver 2.13.20 or higher - Sybase: version 16 as target, capture but not hub Requires Open Client-Library - SingleStore: version 7.1 (7.1.8 or higher) as target, but not as hub or capture - Databricks on Azure: 10.x, 11.x, 12.x, 13.x and 14.x as target, but not as hub or capture Requires ODBC driver manager UnixODBC Requires Simba Spark ODBC Driver 64-bit version 2.6.19 or higher - Databricks on AWS: versions 10.x, 11.x, 12.x, 13.x and 14.x as target, but not as hub or capture Requires ODBC driver manager UnixODBC Requires Simba Spark ODBC Driver 64-bit version 2.6.19 or higher - Google BigQuery: version 2.3 as target, but not as hub or capture Requires ODBC driver manager UnixODBC Requires Simba ODBC Driver for Google BigQuery 64-bit version 2.3.2 or higher - HANA: version 1.0 SPS 11 and 12, and version 2.0 SPS 00 up to SPS 07 Requires ODBC driver manager UnixODBC Requires HANA ODBC driver 2.00 or higher - Teradata: version 16.x and 17.x, as target and hub, not capture Requires Teradata ODBC driver 16.10 for Teradata 16.x Requires Teradata ODBC driver 17.10 for Teradata 17.x Requires TTU 16.10 libraries for Teradata 16.x Requires TTU 17.10 libraries for Teradata 17.x - Greenplum: version 4.0, 4.2, 4.3, 5.5 and 5.10 up to 6.90 as target, not as capture or hub. Requires DataDirect Connect64 XE ODBC 7.1.3.99 - 7.1.5 driver for Greenplum provided by Pivotal. - Redshift: version 1.0.x, as target, not as capture or hub. Requires ODBC driver manager UnixODBC. Requires Amazon Redshift driver 1.2.6.1006-1 - File replication: local, FTP/FTPS and SFTP - Apache Hadoop HDFS: version 3.3.6 and above HVR needs Hadoop 3.3.6+ libraries and Java 8 on its local machine to access HDFS. 'hadoop' command must be in $PATH or Hadoop home must have been configured via one of the following environment variables: $HADOOP_COMMON_HOME, $HADOOP_HOME, $HADOOP_PREFIX. - Apache Hive: version 1.1, 2.1, 2.3, 3.0, 3.1 with UnixODBC driver manager HVR can create Hive external tables above S3/HDFS files. Hvr only uses these during compare. Refresh or integration of this data is done into the S3/HDSF file location. Capture from Hive and hub database in Hive are not supported. Requires ODBC driver manager UnixODBC. Requires HortonWorks ODBC 2.1.2 or above for Hadoop Hive Requires Cloudera ODBC 2.5.12 or above for Hadoop Hive - Apache Kafka: version 0.8 and above (including 0.11) - S3: supported - Google Cloud Storage: supported - Azure Blob FS: supported - Azure Data Lake Store: supported - Installation: Hub and Agent - Perl 5.8 or higher (only needed for hub machine). BUNDLED IN THIS INSTALLATION ------------------------------------- - Fivetran Proxy Agent 1.0.19 - Java Runtime Environment (JRE) 17.52.17 API VERSIONS SUPPORTED BY THIS HUB SERVER AND FOR REMOTE CLI ------------------------------------------------------------ /api/v0 /api/v6.0.5 /api/v6.0.5.1 /api/v6.0.5.2 /api/v6.1 /api/v6.1.0.3 /api/v6.1.0.4 /api/v6.1.0.5 /api/v6.1.0.6 /api/v6.1.0.7 /api/v6.1.0.11 /api/v6.1.0.15 /api/v6.1.0.26 /api/v6.1.0.36 /api/v6.1.5 /api/v6.1.5.2 HOSTED DATABASE SUPPORT POLICY ----------------------------- Often HVR supported source and target locations that can be installed on premises will be hosted by third parties such as cloud providers including (but not limited to) Alibaba, Amazon, Google, Microsoft and others. These hosted systems are generally supported by HVR as sources and targets insofar as they are compatible with their downloadable counterparts. The certification of compatibility is incumbent on the hosting provider, not HVR. Platforms have been separately documented where compatibility is not comparable and HVR has been able to implement interoperability code changes (for example, Amazon Aurora PostgreSQL). If a provider claims compatibility then they must offer the same SQL and API calls and responses that support the features as indicated in the HVR documentation for the corresponding location class requirements in the appropriate source and target sections. In cases where compatibility is not complete then support is not guaranteed. Any issue that may exhibit itself on the hosted platform must be reproducible on an equivalent downloadable version to be considered a bug in the HVR software. INSTALLING A FIVETRAN HVR UPGRADE ------------------------- Often all machines are upgraded at the same time. Current HVR version are fully compatible with two previous major versions; it is not necessary to upgrade all machines in a channel at once. Instead it is possible to only upgrade certain machines. E.g. only the hub machine, agent machine, capture (could be the hub or an agent), or the integrate machine (could be the hub or an agent). If this is done, it should be understood that each HVR release fixes some bugs and/or contains new features. Each bug fix or feature is only effective if the specific machine(s) indicated for it are upgraded. New features should not be used until all machines that are specified for that feature are upgraded, otherwise errors can occur. For example, if a new HVR release fixes an integrate bug, then that release must be installed on the machine(s) which do integrate. So if only the hub machine is upgraded, then there will be no benefit. To decide whether each machines needs to be upgraded, see the descriptions in the release notes below. Each description ends with a line says which machine must be upgraded for that specific bug fix or feature to be effective. LOG BASED CAPTURE SUPPORT ------------------------- - Ingres Log based capture is supported for all Ingres versions supported by HVR. No support for dual or partitioned DBMS log-files. No support for page-spanning rows. DDL replication (using action AdaptDDL) is not supported. - Oracle Log based capture is supported for all Oracle versions supported by HVR. Oracle RAC (cluster) and Oracle ASM are supported. For Oracle version 9, log-based capture is not supported for LOBs or tables without a primary key. Compressed tables are only supported for Oracle 11.2 and higher. Log based capture is supported from Data Guard standby database for Oracle 11 and higher. HVR can also capture from an database that was previously a Data Guard target. If HVR was capturing changes from one primary Oracle database and a role transition occur (so that a different Data Guard target becomes the new primary) then HVR can continue capturing from the new primary, including capturing any changes which occurred before the transition. This process is automatic, providing that the HVR location is connecting to Oracle in a way which 'follows the primary'. Log based capture from tables that are encrypted using Oracle TDE is supported for Oracle version 11 and higher. Both tables in an encrypted tablespace and tables with encrypted columns are supported. HVR supports software and hardware (HSM) wallets. If the wallet is not configured as auto-login (Oracle internal file cwallet.sso), the password for the software wallet or HSM needs to be set on the HVR Live Wallet port using the hvrlivewallet command. The software wallet can be in ASM or in a local filesystem. Capturing from Oracle TDE is not supported on HP-UX. Platforms that use OpenSSL version 3 do not support auto-login wallets for Oracle version 11. - SQL Server Log based capture is supported for SQL Server 2008, 2012, 2014, 2016, 2017, 2019 and 2022. Data Compression is supported. Capture from compressed backup transaction log files is supported. Capture from encrypted backup transaction log files is supported. Capture from memory optimized tables is not supported. Capture from tables with XML_COMPRESSION = ON is not supported. Capture of typed XML columns containing xsd:float and xsd:double values is not supported. Log based capture from databases that are encrypted using the SQL Server TDE is supported. The log based capture is supported only for the databases whose Database Encryption Key (DEK) is protected by Certificates. Log based capture is not supported from databases whose DEK is protected by Asymmetric Key. Column level encryption is not supported. Always Encrypted feature is not supported. - DB2 on Linux, Unix, Windows (LUW) Log based capture is supported for DB2 on Linux, Unix, Windows (LUW). The following table types are supported - regular tables, multidimensional clustering (MDC) tables, insert time clustering (ITC) tables, uncompressed tables, row compressed tables (both static and adaptive) and value compressed tables (both static and adaptive). - DB2 for i Log based capture is supported for DB2 for i - DB2 for z/OS Log based capture for DB2 for z/OS is supported. The following table types are supported - regular tables, compressed tables, partitioned tables, history tables and archive tables. Capture from LOAD is not supported. - PostgreSQL Log based capture is supported for all PostgreSQL versions supported by HVR. - HANA Log based capture is supported on Linux (not Windows) for HANA: version 1.0 SPS 11 and 12, and version 2.0 SPS 00 up to SPS 07 Only column-store tables are supported (no row-store tables). - MySQL and MariaDB Log based capture is supported for all MySQL and MariaDB versions supported by HVR. FIVETRAN HVR NETWORK ENCRYPTION ---------------------- An HVR connection to a remote location can be configured so that all communication over the network is encrypted. For network encryption, HVR uses OpenSSL, which is developed by the OpenSSL Project (http://www.openssl.org). HVR uses OpenSSL version 3.2.1 On UNIX, HVR's encryption will attempt to exploit the entropy (randomness) generation capability supplied by the '/dev/urandom' device. For optimal security, it is recommended that this Operating System option is installed, especially on the machine used to generate public/private key pairs CREDITS ------- - License Agreements, Copyrights, and Notices for the third-party software are listed in the hvr.3rdparty file in HVR_HOME directory NEW FEATURES IN HVR 6.2.0/0 (2024-09-25) ---------------------------------------- [T-763647] DB2 FOR I: ADDED SUPPORT FOR BOOLEAN DATA TYPE IN DB2 FOR I To use this feature, upgrade HVR on all machine(s). HVR now supports the BOOLEAN data type in Db2 for i. Note that some older Db2 for i ODBC drivers have limitations in describing BOOLEAN columns. As a result, HVR will coerce source data to CHAR(1) instead of BOOLEAN. Integrate will still work properly when the source data contains only valid BOOLEAN values, but it may result in: - integer values other than 0 or 1 get silently converted to 1. - a Db2 for i error "SQL0402" for non-integer values. Defining action TableProperties with CoerceErrorPolicy cannot prevent this behavior. [T-777706] POSTGRESQL: DROP SUPPORT FOR POSTGRESQL 9.4 To use this feature, upgrade HVR on all machine(s). This change removes support for PostgreSQL 9.4. PROBLEMS FIXED IN HVR 6.2.0/0 (2024-09-25) ---------------------------------------- [T-752851] FIXED CHANNEL ACTIVATION FAILING WITH F_JR0923 To fix this bug, upgrade HVR on the hub machine(s). Fixed channel activation failing with F_JR0923 even though ColumnProperties with parameter ExpressionScope has parameter Context set, since the Context should only affect refresh/compare, activation shouldn't have been impacted. This fix ensure that context is always considered during the activation checks for ExpressionScope. [T-755955] FIXED F_JX0021 IN SLACK ALERTS CAUSED BY IMPROPER ESCAPING To fix this bug, upgrade HVR on the hub machine(s). Fixed a bug in HVR Slack alerts. In a rare edge case, if the alert was truncated in such a way that left it ending with a backslash, it caused an F_JX0021 error. In addition, the system would fail in creating an event for the alert error because of bad escaping for backslashes and double quotes. This fixes both those problems. [T-761882] FIXED F_JT0D2C WHEN USING OPT-IN BEFORE IMAGE FOR SAP UNPACK To fix this bug, upgrade HVR on the integrate machine(s). Fixed `F_JT0D2C` when using opt-in before image for SAP unpack. Only applies for opt-in cases where `$HVR_SAP_UNPACK_USE_BEFORE` is set. Fixed the sorting of update pairs such that they are kept together when double sorting in some SAP unpack scenarios. [T-762287] IMPROVED DATA COMPARE WITH BOUNDARY SLICING FOR NON-NULLABLE DATATYPE To fix this bug, upgrade HVR on the hub machine(s). Fixed an issue the query of the last slice for data compare with range slicing did try to match for NULL values in its where clause for columns that where non-nullable. For some databases this lead to a full table scan. Now we do not match for NULL values for columns that are not nullable according to the channel definition. If the source database has a nullable key column, and the column is not nullable in the channel definition, the NULL key will now no longer be in any slice when using boundary slicing. [T-776910] FIXED THE DATA TYPE OF HVR_CAP_LOC COLUMN IN HISTORY TABLE To fix this bug, upgrade HVR on the hub machine(s). Fixed history table having hvr_cap_loc column as varchar(5) while it should have it as varchar(12) [T-784478] FIXED BURST TABLES IN CUSTOM SCHEMA ON ORACLE TARGET NOT DROPPING ON DEACTIVATE To fix this bug, upgrade HVR on the integrate machine(s). Fixed burst tables in custom schema on Oracle target not dropping on deactivate [T-784570] FIXED FAILING INTEGRATE ON MYSQL TARGET WITH CUSTOM BURST TABLE SCHEMA DEFINED To fix this bug, upgrade HVR on the integrate machine(s). Fixed failing integrate on MySQL target with Burst integrate and custom burst table schema defined. [T-778939] INGRES: FIXED F_JZ220A IN CASE OF UNEXPECTED TIMESTAMP To fix this bug, upgrade HVR on the capture machine(s). Implemented ZIZ_INGRES_Y2K38_USE_32BIT_TIMESTAMP to fix Ingres F_JZ220A in case of unexpected timestamp. This environment variable is used to choose either to ignore or not ignore the higher 4 bytes in the 8-byte timestamp value captured from Ingres log file. In case of ignore the date is limited by 2038 year. [T-597251] ORACLE: FIXED INVALID DATE WHILE TRYING TO CONVERT DATE TO NUMBER To fix this bug, upgrade HVR on the capture machine(s). Fixed an issue where HVR exception `F_JD20D7: Invalid date encountered while trying to convert date to number of seconds since 01.01.1970 for column` was thrown from an automated test. Issue is fixed now. Issue was introduced by: T-564881 [T-738837] ORACLE: IMPROVED ORACLE CHANGE CAPTURE FOR UNDO CHAINING To fix this bug, upgrade HVR on the capture machine(s). Fixed an issue where Oracle log parsing encountered the error - "F_JZ1122: Chained undo change vector does not have enough structures." This fix allows multiple undo segments to be collated, split blocks to be merged between segments, and ensures proper handling of straddling undo segments. Additionally, the `hvrlogdump` output is now consistent with Oracle's native dumping format. [T-767763] POSTGRESQL: ENHANCED ERROR REPORTING FOR DATABASE SUB-TRANSACTIONS To fix this bug, upgrade HVR on all machine(s). HVR could previously report an F_JD22D4 error without any accompanying details when it occurred within a database sub-transaction. This fix ensures that relevant error details are now included. [T-774367] POSTGRESQL: FIXED ENDING CYCLES IN THE BEGINNING OF CAPTURE To fix this bug, upgrade HVR on the capture machine(s). HVR PostgreSQL capture could fail to end the capture cycles when `HVR_PQ_STREAMING_REPLICATION_RESTART_AT_END_OF_WAL` is `1` which is currently the default for PostgreSQL versions below 16. This change fixes the problem. [T-779065] POSTGRESQL: FIXED ERROR F_JT0921 WHEN ADDING VIEWS TO CHANNEL To fix this bug, upgrade HVR on the hub machine(s). Fixed error F_JT0921 when adding views or materialized views to channel from PostgreSQL. This could be reproduced both on UI and CLI. [T-735358] SQL SERVER: FIXED MSSQL F_JD0A0E ERROR WHEN CAPTURING UPDATE TRANSACTIONS To fix this bug, upgrade HVR on the capture machine(s). Fixed an issue where HVR could fail with error F_JD0A0E when capturing updates for LOB columns, even though those updates had been correctly integrated on the target during a prior capture cycle. [T-717979] SYBASE ASE: FIXED ERROR F_JT0406 WITH TRANSACTION LOG RECORD HANDLING To fix this bug, upgrade HVR on the capture machine(s). Fixed an issue with the handling and pairing of NOOP, INOOP, and DNOOP transaction log records, which are used to form the before and after images of an UPDATE operation. Previously, mishandling these records in some instances caused the following error - "Error Code: F_JT0406 - Scanner encountered 'after update' before the 'before update' change is ready". The fix ensures that DNOOP records are correctly matched by searching backward for their corresponding INOOP records. Additionally, NOOP records are handled properly, canceling the preceding DNOOP records as expected. [T-763583] SYBASE ASE: FIXED ERROR F_JZ270B WHERE THE DELETE RECORD FOR DNOOP WAS NOT FOUND To fix this bug, upgrade HVR on the capture machine(s). Fixed an issue with INSIND record processing during Direct capture, which made adjustments only for After images of UPDATEs and failed to account for Before images that might have been produced earlier, particularly with LOBs and ROWIMAGE records. This issue caused the following error - "F_JZ270B: The DNOOP record at (,) was expected to be matched to a DELETE record with data page and rowid (,), but that data row was not found." This fix ensures that proper adjustments are now made for Before images of UPDATE operations. [T-775721] SYBASE ASE: FIXED REFRESH JOB FOR SYBASE TARGETS WITHOUT CASE-SENSITIVE NAMES. To fix this bug, upgrade HVR on the integrate machine(s). The `Refresh` job was failing for sources with Case-Sensitive Names enabled, and Sybase targets without Case-Sensitive Names. [T-757389] UI: FIXED MANAGED SECRETS OPERATION ON WINDOWS To fix this bug, upgrade HVR on the hub machine(s). Fixed Managed Secrets to use hvrmanagedpassword.bat script name on Windows. [T-775030] UI: FIXED UNEXPECTED ENVIRONMENT ACTIONS APPLIED IN TABLE SELECTION UI To fix this bug, upgrade HVR on the hub machine(s). Fixed unexpected Environment actions being applied when browsing DB schemas on table selection in UI, by passing channel name to the REST API. [T-777475] UI: FIXED REFRESH/COMPARE DIFFERENCE VIEWING PERMISSION REQUIREMENTS To fix this bug, upgrade HVR on the hub machine(s). Changed permission requirements from ReadExecRefresh to ReadExec for querying refresh/compare differences, as compare can be performed by a ReadExec user. [T-779939] UI: FIXED ALERTS NOT SAVING SMTP "SPECIFIC PORT" To fix this bug, upgrade HVR on the hub machine(s). An alert option "Specific Port" under "Advanced SMTP Configuration" section in UI could be cleared (removed, which also means resetting to the default value), if "Advanced SMTP Configuration" section is collapsed. This has been fixed.