Failed jobs do not become expired to allow you to re-queue them without any
time pressure. You should re-queue or delete them manually, or apply AutomaticRetry(OnAttemptsExceeded = AttemptsExceededAction.Delete)
attribute to delete them automatically.
An exception occurred during performance of the job. Fewer details...
23.11.2024 04:26:57
System.InvalidOperationException
Sequence contains more than one element
System.InvalidOperationException: Sequence contains more than one element
at System.Linq.ThrowHelper.ThrowMoreThanOneElementException()
at System.Linq.Enumerable.TryGetSingle[TSource](IEnumerable`1source, Boolean&found)
at lambda_method1952(Closure , QueryContext )
at RuketoAutomation.Service.Background.SynchronizationData.SaveEntity[T](TcrmEntity)
at RuketoAutomation.Service.Background.SynchronizationData.LoadRareDataFromApi[TConvertData](GetDataDelegategetDataDelegate, ConvertDataDelegate`1convertDataDelegate, GetCountPageDelegategetCountPageDelegate)
at RuketoAutomation.Service.Background.SynchronizationData.CrmRareSynchronize()
at RuketoAutomation.Service.Background.SynchronizationData.RareBackgroundService()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Tasktask)
An exception occurred during performance of the job. More details...
23.11.2024 04:28:17
System.InvalidOperationException
Sequence contains more than one element
System.InvalidOperationException: Sequence contains more than one element
at System.Linq.ThrowHelper.ThrowMoreThanOneElementException()
at System.Linq.Enumerable.TryGetSingle[TSource](IEnumerable`1source, Boolean&found)
at lambda_method1952(Closure , QueryContext )
at RuketoAutomation.Service.Background.SynchronizationData.SaveEntity[T](TcrmEntity)
at RuketoAutomation.Service.Background.SynchronizationData.LoadRareDataFromApi[TConvertData](GetDataDelegategetDataDelegate, ConvertDataDelegate`1convertDataDelegate, GetCountPageDelegategetCountPageDelegate)
at RuketoAutomation.Service.Background.SynchronizationData.CrmRareSynchronize()
at RuketoAutomation.Service.Background.SynchronizationData.RareBackgroundService()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Tasktask)
An exception occurred during performance of the job. More details...
23.11.2024 04:21:15
System.InvalidOperationException
Sequence contains more than one element
System.InvalidOperationException: Sequence contains more than one element
at System.Linq.ThrowHelper.ThrowMoreThanOneElementException()
at System.Linq.Enumerable.TryGetSingle[TSource](IEnumerable`1source, Boolean&found)
at lambda_method1952(Closure , QueryContext )
at RuketoAutomation.Service.Background.SynchronizationData.SaveEntity[T](TcrmEntity)
at RuketoAutomation.Service.Background.SynchronizationData.LoadRareDataFromApi[TConvertData](GetDataDelegategetDataDelegate, ConvertDataDelegate`1convertDataDelegate, GetCountPageDelegategetCountPageDelegate)
at RuketoAutomation.Service.Background.SynchronizationData.CrmRareSynchronize()
at RuketoAutomation.Service.Background.SynchronizationData.RareBackgroundService()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Tasktask)
An exception occurred during performance of the job. More details...
23.11.2024 04:22:28
System.InvalidOperationException
Sequence contains more than one element
System.InvalidOperationException: Sequence contains more than one element
at System.Linq.ThrowHelper.ThrowMoreThanOneElementException()
at System.Linq.Enumerable.TryGetSingle[TSource](IEnumerable`1source, Boolean&found)
at lambda_method1952(Closure , QueryContext )
at RuketoAutomation.Service.Background.SynchronizationData.SaveEntity[T](TcrmEntity)
at RuketoAutomation.Service.Background.SynchronizationData.LoadRareDataFromApi[TConvertData](GetDataDelegategetDataDelegate, ConvertDataDelegate`1convertDataDelegate, GetCountPageDelegategetCountPageDelegate)
at RuketoAutomation.Service.Background.SynchronizationData.CrmRareSynchronize()
at RuketoAutomation.Service.Background.SynchronizationData.RareBackgroundService()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Tasktask)
An exception occurred during performance of the job. More details...
23.11.2024 04:21:30
System.InvalidOperationException
Sequence contains more than one element
System.InvalidOperationException: Sequence contains more than one element
at System.Linq.ThrowHelper.ThrowMoreThanOneElementException()
at System.Linq.Enumerable.TryGetSingle[TSource](IEnumerable`1source, Boolean&found)
at lambda_method1952(Closure , QueryContext )
at RuketoAutomation.Service.Background.SynchronizationData.SaveEntity[T](TcrmEntity)
at RuketoAutomation.Service.Background.SynchronizationData.LoadRareDataFromApi[TConvertData](GetDataDelegategetDataDelegate, ConvertDataDelegate`1convertDataDelegate, GetCountPageDelegategetCountPageDelegate)
at RuketoAutomation.Service.Background.SynchronizationData.CrmRareSynchronize()
at RuketoAutomation.Service.Background.SynchronizationData.RareBackgroundService()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Tasktask)
An exception occurred during performance of the job. More details...
23.11.2024 04:22:44
System.InvalidOperationException
Sequence contains more than one element
System.InvalidOperationException: Sequence contains more than one element
at System.Linq.ThrowHelper.ThrowMoreThanOneElementException()
at System.Linq.Enumerable.TryGetSingle[TSource](IEnumerable`1source, Boolean&found)
at lambda_method1952(Closure , QueryContext )
at RuketoAutomation.Service.Background.SynchronizationData.SaveEntity[T](TcrmEntity)
at RuketoAutomation.Service.Background.SynchronizationData.LoadRareDataFromApi[TConvertData](GetDataDelegategetDataDelegate, ConvertDataDelegate`1convertDataDelegate, GetCountPageDelegategetCountPageDelegate)
at RuketoAutomation.Service.Background.SynchronizationData.CrmRareSynchronize()
at RuketoAutomation.Service.Background.SynchronizationData.RareBackgroundService()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Tasktask)
An exception occurred during performance of the job. More details...
23.11.2024 04:20:30
System.InvalidOperationException
Sequence contains more than one element
System.InvalidOperationException: Sequence contains more than one element
at System.Linq.ThrowHelper.ThrowMoreThanOneElementException()
at System.Linq.Enumerable.TryGetSingle[TSource](IEnumerable`1source, Boolean&found)
at lambda_method1952(Closure , QueryContext )
at RuketoAutomation.Service.Background.SynchronizationData.SaveEntity[T](TcrmEntity)
at RuketoAutomation.Service.Background.SynchronizationData.LoadRareDataFromApi[TConvertData](GetDataDelegategetDataDelegate, ConvertDataDelegate`1convertDataDelegate, GetCountPageDelegategetCountPageDelegate)
at RuketoAutomation.Service.Background.SynchronizationData.CrmRareSynchronize()
at RuketoAutomation.Service.Background.SynchronizationData.RareBackgroundService()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Tasktask)
An exception occurred during performance of the job. More details...
23.11.2024 04:20:41
System.InvalidOperationException
Sequence contains more than one element
System.InvalidOperationException: Sequence contains more than one element
at System.Linq.ThrowHelper.ThrowMoreThanOneElementException()
at System.Linq.Enumerable.TryGetSingle[TSource](IEnumerable`1source, Boolean&found)
at lambda_method1952(Closure , QueryContext )
at RuketoAutomation.Service.Background.SynchronizationData.SaveEntity[T](TcrmEntity)
at RuketoAutomation.Service.Background.SynchronizationData.LoadRareDataFromApi[TConvertData](GetDataDelegategetDataDelegate, ConvertDataDelegate`1convertDataDelegate, GetCountPageDelegategetCountPageDelegate)
at RuketoAutomation.Service.Background.SynchronizationData.CrmRareSynchronize()
at RuketoAutomation.Service.Background.SynchronizationData.RareBackgroundService()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Tasktask)
An exception occurred during performance of the job. More details...
23.11.2024 04:15:41
System.Threading.Tasks.TaskCanceledException
The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
---> System.TimeoutException: A task was canceled.
---> System.Threading.Tasks.TaskCanceledException: A task was canceled.
at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellation(CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.GetHttp11ConnectionAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessagerequest, Booleanasync, BooleandoRequestAuth, CancellationTokencancellationToken)
at System.Net.Http.HttpMessageHandlerStage.Send(HttpRequestMessagerequest, CancellationTokencancellationToken)
at System.Net.Http.DiagnosticsHandler.SendAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpMessageHandlerStage.Send(HttpRequestMessagerequest, CancellationTokencancellationToken)
at System.Net.Http.SocketsHttpHandler.Send(HttpRequestMessagerequest, CancellationTokencancellationToken)
at System.Net.Http.HttpClientHandler.Send(HttpRequestMessagerequest, CancellationTokencancellationToken)
at System.Net.Http.HttpMessageInvoker.Send(HttpRequestMessagerequest, CancellationTokencancellationToken)
at System.Net.Http.HttpClient.Send(HttpRequestMessagerequest, HttpCompletionOptioncompletionOption, CancellationTokencancellationToken)
--- End of inner exception stack trace ---
--- End of inner exception stack trace ---
at System.Net.Http.HttpClient.HandleFailure(Exceptione, BooleantelemetryStarted, HttpResponseMessageresponse, CancellationTokenSourcects, CancellationTokencancellationToken, CancellationTokenSourcependingRequestsCts)
at System.Net.Http.HttpClient.Send(HttpRequestMessagerequest, HttpCompletionOptioncompletionOption, CancellationTokencancellationToken)
at RuketoAutomation.Service.Crm.CrmApiWorker.GetData(Stringaddress, Stringparameters)
at RuketoAutomation.Service.Crm.CrmApiWorker.GetPackagesData(Int32page)
at RuketoAutomation.Service.Background.SynchronizationData.LoadRareDataFromApi[TConvertData](GetDataDelegategetDataDelegate, ConvertDataDelegate`1convertDataDelegate, GetCountPageDelegategetCountPageDelegate)
at RuketoAutomation.Service.Background.SynchronizationData.CrmRareSynchronize()
at RuketoAutomation.Service.Background.SynchronizationData.RareBackgroundService()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Tasktask)
An exception occurred during performance of the job. More details...
23.11.2024 04:18:26
System.Threading.Tasks.TaskCanceledException
The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
---> System.TimeoutException: A task was canceled.
---> System.Threading.Tasks.TaskCanceledException: A task was canceled.
at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellation(CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.GetHttp11ConnectionAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessagerequest, Booleanasync, BooleandoRequestAuth, CancellationTokencancellationToken)
at System.Net.Http.HttpMessageHandlerStage.Send(HttpRequestMessagerequest, CancellationTokencancellationToken)
at System.Net.Http.DiagnosticsHandler.SendAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpMessageHandlerStage.Send(HttpRequestMessagerequest, CancellationTokencancellationToken)
at System.Net.Http.SocketsHttpHandler.Send(HttpRequestMessagerequest, CancellationTokencancellationToken)
at System.Net.Http.HttpClientHandler.Send(HttpRequestMessagerequest, CancellationTokencancellationToken)
at System.Net.Http.HttpMessageInvoker.Send(HttpRequestMessagerequest, CancellationTokencancellationToken)
at System.Net.Http.HttpClient.Send(HttpRequestMessagerequest, HttpCompletionOptioncompletionOption, CancellationTokencancellationToken)
--- End of inner exception stack trace ---
--- End of inner exception stack trace ---
at System.Net.Http.HttpClient.HandleFailure(Exceptione, BooleantelemetryStarted, HttpResponseMessageresponse, CancellationTokenSourcects, CancellationTokencancellationToken, CancellationTokenSourcependingRequestsCts)
at System.Net.Http.HttpClient.Send(HttpRequestMessagerequest, HttpCompletionOptioncompletionOption, CancellationTokencancellationToken)
at RuketoAutomation.Service.Crm.CrmApiWorker.GetData(Stringaddress, Stringparameters)
at RuketoAutomation.Service.Crm.CrmApiWorker.GetPackagesData(Int32page)
at RuketoAutomation.Service.Background.SynchronizationData.LoadRareDataFromApi[TConvertData](GetDataDelegategetDataDelegate, ConvertDataDelegate`1convertDataDelegate, GetCountPageDelegategetCountPageDelegate)
at RuketoAutomation.Service.Background.SynchronizationData.CrmRareSynchronize()
at RuketoAutomation.Service.Background.SynchronizationData.RareBackgroundService()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Tasktask)