SynchronizationData.RareBackgroundService

// Id: #961574
using RuketoAutomation.Service.Background;

var synchronizationData = Activate<SynchronizationData>();
await synchronizationData.RareBackgroundService();

Parameters

CurrentCulture
"ru-RU"
CurrentUICulture
"en-US"
RecurringJobId
"rareJob"
RetryCount
10
Time
1733960768

State

12.12.2024 04:21:51 (+2.792s) Failed

An exception occurred during performance of the job.

System.InvalidOperationException (e6e5b76a-e2f6-405c-81f3-860482fc3ddd)

Sequence contains more than one element

System.InvalidOperationException: Sequence contains more than one element
   at System.Linq.ThrowHelper.ThrowMoreThanOneElementException()
   at System.Linq.Enumerable.TryGetSingle[TSource](IEnumerable`1 source, Boolean& found)
   at lambda_method1510(Closure , QueryContext )
   at RuketoAutomation.Service.Background.SynchronizationData.SaveEntity[T](T crmEntity) in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 451
   at RuketoAutomation.Service.Background.SynchronizationData.LoadRareDataFromApi[TConvertData](GetDataDelegate getDataDelegate, ConvertDataDelegate`1 convertDataDelegate, GetCountPageDelegate getCountPageDelegate) in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 421
   at RuketoAutomation.Service.Background.SynchronizationData.LoadProducts(CrmApiWorker crmApiWorker, JsonWorker jsonWorker) in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 387
   at RuketoAutomation.Service.Background.SynchronizationData.CrmRareSynchronize() in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 368
   at RuketoAutomation.Service.Background.SynchronizationData.RareBackgroundService() in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 346
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+3ms Processing

Server:
e6e5b76a-e2f6-405c-81f3-860482fc3ddd
Worker:
53185cf4

+1h 53m Enqueued

Triggered by DelayedJobScheduler

+ <1ms Scheduled

Retry attempt 10 of 10: Sequence contains more than one element

Enqueue at:
12.12.2024 04:21:45

+2.743s Failed

An exception occurred during performance of the job.

System.InvalidOperationException (e6e5b76a-e2f6-405c-81f3-860482fc3ddd)

Sequence contains more than one element

System.InvalidOperationException: Sequence contains more than one element
   at System.Linq.ThrowHelper.ThrowMoreThanOneElementException()
   at System.Linq.Enumerable.TryGetSingle[TSource](IEnumerable`1 source, Boolean& found)
   at lambda_method1510(Closure , QueryContext )
   at RuketoAutomation.Service.Background.SynchronizationData.SaveEntity[T](T crmEntity) in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 451
   at RuketoAutomation.Service.Background.SynchronizationData.LoadRareDataFromApi[TConvertData](GetDataDelegate getDataDelegate, ConvertDataDelegate`1 convertDataDelegate, GetCountPageDelegate getCountPageDelegate) in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 421
   at RuketoAutomation.Service.Background.SynchronizationData.LoadProducts(CrmApiWorker crmApiWorker, JsonWorker jsonWorker) in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 387
   at RuketoAutomation.Service.Background.SynchronizationData.CrmRareSynchronize() in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 368
   at RuketoAutomation.Service.Background.SynchronizationData.RareBackgroundService() in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 346
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+4ms Processing

Server:
e6e5b76a-e2f6-405c-81f3-860482fc3ddd
Worker:
2d5f1520

+1h 9m Enqueued

Triggered by DelayedJobScheduler

+ <1ms Scheduled

Retry attempt 9 of 10: Sequence contains more than one element

Enqueue at:
12.12.2024 02:28:05

+4.859s Failed

An exception occurred during performance of the job.

System.InvalidOperationException (e6e5b76a-e2f6-405c-81f3-860482fc3ddd)

Sequence contains more than one element

System.InvalidOperationException: Sequence contains more than one element
   at System.Linq.ThrowHelper.ThrowMoreThanOneElementException()
   at System.Linq.Enumerable.TryGetSingle[TSource](IEnumerable`1 source, Boolean& found)
   at lambda_method1510(Closure , QueryContext )
   at RuketoAutomation.Service.Background.SynchronizationData.SaveEntity[T](T crmEntity) in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 451
   at RuketoAutomation.Service.Background.SynchronizationData.LoadRareDataFromApi[TConvertData](GetDataDelegate getDataDelegate, ConvertDataDelegate`1 convertDataDelegate, GetCountPageDelegate getCountPageDelegate) in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 421
   at RuketoAutomation.Service.Background.SynchronizationData.LoadProducts(CrmApiWorker crmApiWorker, JsonWorker jsonWorker) in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 387
   at RuketoAutomation.Service.Background.SynchronizationData.CrmRareSynchronize() in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 368
   at RuketoAutomation.Service.Background.SynchronizationData.RareBackgroundService() in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 346
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+4ms Processing

Server:
e6e5b76a-e2f6-405c-81f3-860482fc3ddd
Worker:
f0247dfe

+42m 41.727s Enqueued

Triggered by DelayedJobScheduler

+ <1ms Scheduled

Retry attempt 8 of 10: Sequence contains more than one element

Enqueue at:
12.12.2024 01:18:35

+5.431s Failed

An exception occurred during performance of the job.

System.InvalidOperationException (e6e5b76a-e2f6-405c-81f3-860482fc3ddd)

Sequence contains more than one element

System.InvalidOperationException: Sequence contains more than one element
   at System.Linq.ThrowHelper.ThrowMoreThanOneElementException()
   at System.Linq.Enumerable.TryGetSingle[TSource](IEnumerable`1 source, Boolean& found)
   at lambda_method1510(Closure , QueryContext )
   at RuketoAutomation.Service.Background.SynchronizationData.SaveEntity[T](T crmEntity) in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 451
   at RuketoAutomation.Service.Background.SynchronizationData.LoadRareDataFromApi[TConvertData](GetDataDelegate getDataDelegate, ConvertDataDelegate`1 convertDataDelegate, GetCountPageDelegate getCountPageDelegate) in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 421
   at RuketoAutomation.Service.Background.SynchronizationData.LoadProducts(CrmApiWorker crmApiWorker, JsonWorker jsonWorker) in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 387
   at RuketoAutomation.Service.Background.SynchronizationData.CrmRareSynchronize() in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 368
   at RuketoAutomation.Service.Background.SynchronizationData.RareBackgroundService() in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 346
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+4ms Processing

Server:
e6e5b76a-e2f6-405c-81f3-860482fc3ddd
Worker:
e0a68bdd

+23m 57.367s Enqueued

Triggered by DelayedJobScheduler

+ <1ms Scheduled

Retry attempt 7 of 10: Sequence contains more than one element

Enqueue at:
12.12.2024 00:35:43

+3.841s Failed

An exception occurred during performance of the job.

System.InvalidOperationException (e6e5b76a-e2f6-405c-81f3-860482fc3ddd)

Sequence contains more than one element

System.InvalidOperationException: Sequence contains more than one element
   at System.Linq.ThrowHelper.ThrowMoreThanOneElementException()
   at System.Linq.Enumerable.TryGetSingle[TSource](IEnumerable`1 source, Boolean& found)
   at lambda_method1510(Closure , QueryContext )
   at RuketoAutomation.Service.Background.SynchronizationData.SaveEntity[T](T crmEntity) in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 451
   at RuketoAutomation.Service.Background.SynchronizationData.LoadRareDataFromApi[TConvertData](GetDataDelegate getDataDelegate, ConvertDataDelegate`1 convertDataDelegate, GetCountPageDelegate getCountPageDelegate) in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 421
   at RuketoAutomation.Service.Background.SynchronizationData.LoadProducts(CrmApiWorker crmApiWorker, JsonWorker jsonWorker) in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 387
   at RuketoAutomation.Service.Background.SynchronizationData.CrmRareSynchronize() in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 368
   at RuketoAutomation.Service.Background.SynchronizationData.RareBackgroundService() in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 346
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+4ms Processing

Server:
e6e5b76a-e2f6-405c-81f3-860482fc3ddd
Worker:
28c047fa

+12m 11.677s Enqueued

Triggered by DelayedJobScheduler

+ <1ms Scheduled

Retry attempt 6 of 10: Sequence contains more than one element

Enqueue at:
12.12.2024 00:11:42

+3.942s Failed

An exception occurred during performance of the job.

System.InvalidOperationException (e6e5b76a-e2f6-405c-81f3-860482fc3ddd)

Sequence contains more than one element

System.InvalidOperationException: Sequence contains more than one element
   at System.Linq.ThrowHelper.ThrowMoreThanOneElementException()
   at System.Linq.Enumerable.TryGetSingle[TSource](IEnumerable`1 source, Boolean& found)
   at lambda_method1510(Closure , QueryContext )
   at RuketoAutomation.Service.Background.SynchronizationData.SaveEntity[T](T crmEntity) in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 451
   at RuketoAutomation.Service.Background.SynchronizationData.LoadRareDataFromApi[TConvertData](GetDataDelegate getDataDelegate, ConvertDataDelegate`1 convertDataDelegate, GetCountPageDelegate getCountPageDelegate) in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 421
   at RuketoAutomation.Service.Background.SynchronizationData.LoadProducts(CrmApiWorker crmApiWorker, JsonWorker jsonWorker) in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 387
   at RuketoAutomation.Service.Background.SynchronizationData.CrmRareSynchronize() in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 368
   at RuketoAutomation.Service.Background.SynchronizationData.RareBackgroundService() in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 346
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+3ms Processing

Server:
e6e5b76a-e2f6-405c-81f3-860482fc3ddd
Worker:
91175016

+6m 56.414s Enqueued

Triggered by DelayedJobScheduler

+ <1ms Scheduled

Retry attempt 5 of 10: Sequence contains more than one element

Enqueue at:
11.12.2024 23:59:30

+3.904s Failed

An exception occurred during performance of the job.

System.InvalidOperationException (e6e5b76a-e2f6-405c-81f3-860482fc3ddd)

Sequence contains more than one element

System.InvalidOperationException: Sequence contains more than one element
   at System.Linq.ThrowHelper.ThrowMoreThanOneElementException()
   at System.Linq.Enumerable.TryGetSingle[TSource](IEnumerable`1 source, Boolean& found)
   at lambda_method1510(Closure , QueryContext )
   at RuketoAutomation.Service.Background.SynchronizationData.SaveEntity[T](T crmEntity) in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 451
   at RuketoAutomation.Service.Background.SynchronizationData.LoadRareDataFromApi[TConvertData](GetDataDelegate getDataDelegate, ConvertDataDelegate`1 convertDataDelegate, GetCountPageDelegate getCountPageDelegate) in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 421
   at RuketoAutomation.Service.Background.SynchronizationData.LoadProducts(CrmApiWorker crmApiWorker, JsonWorker jsonWorker) in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 387
   at RuketoAutomation.Service.Background.SynchronizationData.CrmRareSynchronize() in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 368
   at RuketoAutomation.Service.Background.SynchronizationData.RareBackgroundService() in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 346
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+3ms Processing

Server:
e6e5b76a-e2f6-405c-81f3-860482fc3ddd
Worker:
2d5f1520

+3m 12.066s Enqueued

Triggered by DelayedJobScheduler

+ <1ms Scheduled

Retry attempt 4 of 10: Sequence contains more than one element

Enqueue at:
11.12.2024 23:52:28

+3.081s Failed

An exception occurred during performance of the job.

System.InvalidOperationException (e6e5b76a-e2f6-405c-81f3-860482fc3ddd)

Sequence contains more than one element

System.InvalidOperationException: Sequence contains more than one element
   at System.Linq.ThrowHelper.ThrowMoreThanOneElementException()
   at System.Linq.Enumerable.TryGetSingle[TSource](IEnumerable`1 source, Boolean& found)
   at lambda_method1510(Closure , QueryContext )
   at RuketoAutomation.Service.Background.SynchronizationData.SaveEntity[T](T crmEntity) in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 451
   at RuketoAutomation.Service.Background.SynchronizationData.LoadRareDataFromApi[TConvertData](GetDataDelegate getDataDelegate, ConvertDataDelegate`1 convertDataDelegate, GetCountPageDelegate getCountPageDelegate) in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 421
   at RuketoAutomation.Service.Background.SynchronizationData.LoadProducts(CrmApiWorker crmApiWorker, JsonWorker jsonWorker) in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 387
   at RuketoAutomation.Service.Background.SynchronizationData.CrmRareSynchronize() in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 368
   at RuketoAutomation.Service.Background.SynchronizationData.RareBackgroundService() in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 346
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+5ms Processing

Server:
e6e5b76a-e2f6-405c-81f3-860482fc3ddd
Worker:
edfd3c7f

+1m 11.109s Enqueued

Triggered by DelayedJobScheduler

+ <1ms Scheduled

Retry attempt 3 of 10: Sequence contains more than one element

Enqueue at:
11.12.2024 23:49:24

+3.938s Failed

An exception occurred during performance of the job.

System.InvalidOperationException (e6e5b76a-e2f6-405c-81f3-860482fc3ddd)

Sequence contains more than one element

System.InvalidOperationException: Sequence contains more than one element
   at System.Linq.ThrowHelper.ThrowMoreThanOneElementException()
   at System.Linq.Enumerable.TryGetSingle[TSource](IEnumerable`1 source, Boolean& found)
   at lambda_method1510(Closure , QueryContext )
   at RuketoAutomation.Service.Background.SynchronizationData.SaveEntity[T](T crmEntity) in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 451
   at RuketoAutomation.Service.Background.SynchronizationData.LoadRareDataFromApi[TConvertData](GetDataDelegate getDataDelegate, ConvertDataDelegate`1 convertDataDelegate, GetCountPageDelegate getCountPageDelegate) in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 421
   at RuketoAutomation.Service.Background.SynchronizationData.LoadProducts(CrmApiWorker crmApiWorker, JsonWorker jsonWorker) in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 387
   at RuketoAutomation.Service.Background.SynchronizationData.CrmRareSynchronize() in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 368
   at RuketoAutomation.Service.Background.SynchronizationData.RareBackgroundService() in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 346
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+3ms Processing

Server:
e6e5b76a-e2f6-405c-81f3-860482fc3ddd
Worker:
e0a68bdd

+1m 12.098s Enqueued

Triggered by DelayedJobScheduler

+1ms Scheduled

Retry attempt 2 of 10: Sequence contains more than one element

Enqueue at:
11.12.2024 23:48:00

+2.927s Failed

An exception occurred during performance of the job.

System.InvalidOperationException (e6e5b76a-e2f6-405c-81f3-860482fc3ddd)

Sequence contains more than one element

System.InvalidOperationException: Sequence contains more than one element
   at System.Linq.ThrowHelper.ThrowMoreThanOneElementException()
   at System.Linq.Enumerable.TryGetSingle[TSource](IEnumerable`1 source, Boolean& found)
   at lambda_method1510(Closure , QueryContext )
   at RuketoAutomation.Service.Background.SynchronizationData.SaveEntity[T](T crmEntity) in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 451
   at RuketoAutomation.Service.Background.SynchronizationData.LoadRareDataFromApi[TConvertData](GetDataDelegate getDataDelegate, ConvertDataDelegate`1 convertDataDelegate, GetCountPageDelegate getCountPageDelegate) in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 421
   at RuketoAutomation.Service.Background.SynchronizationData.LoadProducts(CrmApiWorker crmApiWorker, JsonWorker jsonWorker) in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 387
   at RuketoAutomation.Service.Background.SynchronizationData.CrmRareSynchronize() in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 368
   at RuketoAutomation.Service.Background.SynchronizationData.RareBackgroundService() in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 346
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+3ms Processing

Server:
e6e5b76a-e2f6-405c-81f3-860482fc3ddd
Worker:
91175016

+44.275s Enqueued

Triggered by DelayedJobScheduler

+ <1ms Scheduled

Retry attempt 1 of 10: Sequence contains more than one element

Enqueue at:
11.12.2024 23:46:45

+2.877s Failed

An exception occurred during performance of the job.

System.InvalidOperationException (e6e5b76a-e2f6-405c-81f3-860482fc3ddd)

Sequence contains more than one element

System.InvalidOperationException: Sequence contains more than one element
   at System.Linq.ThrowHelper.ThrowMoreThanOneElementException()
   at System.Linq.Enumerable.TryGetSingle[TSource](IEnumerable`1 source, Boolean& found)
   at lambda_method1510(Closure , QueryContext )
   at RuketoAutomation.Service.Background.SynchronizationData.SaveEntity[T](T crmEntity) in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 451
   at RuketoAutomation.Service.Background.SynchronizationData.LoadRareDataFromApi[TConvertData](GetDataDelegate getDataDelegate, ConvertDataDelegate`1 convertDataDelegate, GetCountPageDelegate getCountPageDelegate) in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 421
   at RuketoAutomation.Service.Background.SynchronizationData.LoadProducts(CrmApiWorker crmApiWorker, JsonWorker jsonWorker) in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 387
   at RuketoAutomation.Service.Background.SynchronizationData.CrmRareSynchronize() in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 368
   at RuketoAutomation.Service.Background.SynchronizationData.RareBackgroundService() in /home/developer/code/build/RuketoAutomation/Service/Background/SynchronizationData.cs:line 346
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

+5ms Processing

Server:
e6e5b76a-e2f6-405c-81f3-860482fc3ddd
Worker:
28997c3c

+1ms Enqueued

Triggered by recurring job scheduler

11.12.2024 23:46:08 Created