• python
  • javascript
  • reactjs
  • sql
  • c#
  • java
Facebook Twitter Instagram
Devs Fixed
  • python
  • javascript
  • reactjs
  • sql
  • c#
  • java
Devs Fixed
Home ยป Resolved: Debezium SQL Server Source Connectors streams events only once per day

Resolved: Debezium SQL Server Source Connectors streams events only once per day

0
By Isaac Tonny on 16/06/2022 Issue
Share
Facebook Twitter LinkedIn

Question:

I am facing a problem where my Debezium SQL Server Source Connectors is not streaming CDC captures in real time but streams all events once per day at the same time 19:07:29. Below is a sample record
You see the difference in source.ts_ms and ts_ms . source.ts_ms is 15 June 2022 16:47:53.757 but ts_ms is 15 June 2022 19:07:29.349 . The documentation mentions that it means there is a lag but doesnt give any more info.
https://debezium.io/documentation/reference/connectors/sqlserver#sqlserver-change-event-values

In the source object, ts_ms indicates the time when a change was committed in the database. By comparing the value for payload.source.ts_ms with the value for payload.ts_ms, you can determine the lag between the source database update and Debezium.


Below is the connector config

Answer:

Hassan , without logs this is difficult to solve. Please check the following and post more logs
  1. Check if the CDC is enabled on the tables
  2. Query the CDC table in SQL Server , There is a default cdc schema where changes are captured.

  1. The only SMT is a case change , so expect not to have any issues there

  2. Check the Kafka connect logs. From the question probably is getting deployed to k8s. Login into the POD


  1. Do you have any other hints on the logs ?

  2. Are there issues in the SQL Server ? sp_who2 , sp_whoisactive perhaps for any clue ?

  3. Any network issues between kubernetes and the RDS SQL Server ?


While the time correlation may be valid , these would be the ways to find any issues and narrow down what you face

If you have better answer, please add a comment about this, thank you!

debezium sql-server
Share. Facebook Twitter LinkedIn

Related Posts

Resolved: Why reference in pointer array doesn’t have data?

24/03/2023

Resolved: EntityFramework creates/runs migrations using parameterless DataContext instance

24/03/2023

Resolved: Visual Studio 2022 crashes when using breakpoints

24/03/2023

Leave A Reply

© 2023 DEVSFIX.COM

Type above and press Enter to search. Press Esc to cancel.