Background Thread Unhandled AccessViolationException

crazypit · Post by **crazypit** » Fri 01 Jul 2011 07:28

Hello,

At the system described in http://www.devart.com/forums/viewtopic.php?t=20504 and using version 2.20.21 of LinqConnect i still get an AccessViolationException that crashes my CRM server.

Application: Exis.eXray.AppServerService.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.AccessViolationException
Stack:
at Devart.Data.Linq.Provider.i+a.g()
at Devart.Data.Linq.Provider.i+a.b(Devart.Data.Linq.Provider.IConnectionUser)
at Devart.Data.Linq.Provider.DataProvider.ExecuteQuery(CompiledQuery, System.Object[], System.Object[], System.Object)
at Devart.Data.Linq.Provider.DataProvider.ExecuteAllQueries(CompiledQuery, System.Object[])
at Devart.Data.Linq.CompiledQuery.Invoke[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System.Int64, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]](System.__Canon, Int64)
at Exis.eXray.WorkflowEngine.WorkflowService.ProcessCaseInt(Int64)
at Exis.eXray.WorkflowEngine.WorkflowService.ProcessCase(Int64)

ALL exceptions are handled by my code effectively in ProcessCase(Int64) function.

This is another one that happened a few days before:

Application: Exis.eXray.AppServerService.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.AccessViolationException
Stack:
at OciDynamicType.nativeOCILobGetLength(System.Runtime.InteropServices.HandleRef, System.Runtime.InteropServices.HandleRef, IntPtr, Int32 ByRef)
at OciDynamicType.OCILobGetLength(System.Runtime.InteropServices.HandleRef, System.Runtime.InteropServices.HandleRef, IntPtr, Int32 ByRef)
at Devart.Data.Oracle.i.k()
at Devart.Data.Oracle.OracleLob.ReadLobCache()
at Devart.Data.Oracle.i.b()
at Devart.Data.Oracle.as.q(Byte[], Int32, Int32)
at DynamicClass.lambda_method(System.Runtime.CompilerServices.Closure, Devart.Data.Linq.Provider.b, Int32[])
at Devart.Data.Linq.Provider.EntityReader`3[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System.Int64, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].b()
at Devart.Data.Linq.Provider.ObjectReader`1[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].MoveNext()
at System.Collections.Generic.List`1[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]]..ctor(System.Collections.Generic.IEnumerable`1)
at System.Linq.Enumerable.ToList[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]](System.Collections.Generic.IEnumerable`1)
at Exis.eXray.WorkflowEngine.WorkflowService.ExecuteEventAction(Exis.eXray.Domain.Event, Exis.eXray.Domain.Case, Exis.eXray.Domain.EventTypeAction)
at Exis.eXray.WorkflowEngine.WorkflowService.ExecuteMainAction(Exis.eXray.Domain.Event, Exis.eXray.Domain.Case)
at Exis.eXray.WorkflowEngine.WorkflowService.ProcessWorkflow(Exis.eXray.Domain.Case, Exis.eXray.Domain.Event)
at Exis.eXray.WorkflowEngine.WorkflowService.ProcessCaseInt(Int64)
at Exis.eXray.WorkflowEngine.WorkflowService.ProcessCaseOneWay(Int64)

crazypit · Post by **crazypit** » Tue 05 Jul 2011 06:35

And an other yesterday:

Application: Exis.eXray.AppServerService.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.AccessViolationException
Stack:
at Devart.Data.Oracle.OracleDataReader.Close()
at Devart.Data.Linq.Provider.ObjectReader`1[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].Dispose()
at System.Collections.Generic.List`1[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]]..ctor(System.Collections.Generic.IEnumerable`1)
at System.Linq.Enumerable.ToList[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]](System.Collections.Generic.IEnumerable`1)
at Exis.eXray.ClientsEngine.ClientsService.AgentCenter_GetUserAppointments(System.String, Int64, System.Collections.Generic.Dictionary`2 ByRef)

StanislavK · Post by **StanislavK** » Tue 05 Jul 2011 15:52

Sorry for the delay.

Am I correct supposing that you are using the OCI connection mode? If yes, please specify the exact versions of the Oracle server and client you are using. A probable reason of these issues is a problem in the Oracle client software or client/server compatibility. To check if this is the case, you can try switching to the Direct mode or changing the version of the client. Please tell us if these problems occur again in either case.

crazypit · Post by **crazypit** » Wed 06 Jul 2011 07:59

All the information you need are present at the links provided in my first post. The first link points to an older post with info about the system.

Nevertheless, Direct mode cannot be used as this is a RAC system which is unsupported. Also, it is not so easy to just change the Oracle client on a production system!

In any case, the occasional error is not my problem. A proper application should handle these gracefully. My problem is that these errors are not propagated to my main thread and my whole CRM server goes down! Don't you think that all DB access related exceptions should be caught and handled??? This is my real problem! i don't really care about the occasional hickup. I need you to catch all these exceptions and send it to main thread.

StanislavK · Post by **StanislavK** » Wed 06 Jul 2011 14:50

We will prepare you a test build, which should help to localize the problem, in several days.

StanislavK · Post by **StanislavK** » Fri 29 Jul 2011 13:23

We've recently performed some internal fixes that may be related to this issue. Please try using the latest build of LinqConnect or dotConnect for Oracle, and tell us if you encounter the problem again.

crazypit · Post by **crazypit** » Tue 13 Sep 2011 09:17

Using version 2.50.30 of LinqConnect, i still get a lot of System.AccessViolationException errors with the stack:
at Devart.Data.Linq.Provider.ObjectReader`1[[System.Decimal, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].k()
at Devart.Data.Linq.Provider.k+a.d()
at Devart.Data.Linq.Provider.k.e()
at Devart.Data.Linq.Provider.DataProvider.Dispose(Boolean)
at Devart.Data.Linq.DataContext.Dispose()

Faulting module is OraOCIEI11.dll of my Instant Client installation.

Nowadays, i get 20 errors a day that crash my production application server and severely hinders functionality! It seems that the number of crashes is analogous to the work load of the server. In a few months, i will have serious problems getting my system not turned off by my clients! I need help!

StanislavK · Post by **StanislavK** » Thu 15 Sep 2011 15:41

We are investigating the situation, and working on reproducing this problem. We will post here about the results as soon as possible.

crazypit · Post by **crazypit** » Mon 19 Sep 2011 06:58

I have an unusual observation to make that may help with the situation. Apparently the error emerges when i try to dispose a datacontext. My application server hosts 15 WCF services and every call to their functions, opens a new datacontext and finally disposes it to the the pool. The really odd thing is that ONLY A SINGLE function call errors all the time. The stack is :
Stack:
at Devart.Data.Linq.Provider.ObjectReader`1[[System.Decimal, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].k()
at Devart.Data.Linq.Provider.k+a.d()
at Devart.Data.Linq.Provider.k.e()
at Devart.Data.Linq.Provider.DataProvider.Dispose(Boolean)
at Devart.Data.Linq.DataContext.Dispose()
at Exis.eXray.WorkflowEngine.WorkflowService.ProcessCaseInt(Int64)
at Exis.eXray.WorkflowEngine.WorkflowService.ProcessCaseOneWay(Int64)

Can you make something out of this? Is there a way to help you by providing the code of this function? There are a number of WCF timeouts when the clients call this function. The reliable session times out on the server, do you think this has something to do with the problem? An other characteristic is that this function is marked as a OneWay WCF function. I do not know if any of this info is helpful but it is really odd that the same function errors in datacontext disposal. Code of the relevant function follows:

public void ProcessCaseOneWay(long CaseId)
{
try
{
ProcessCaseInt(CaseId);
}
catch (Exception)
{

}
}

private void ProcessCaseInt(long CaseId)
{
try
{
LockMutex = new Mutex(false, "Exis.eXray.WorkflowEngine." + ServicePort + ".Mutex." + CaseId);
if (!LockMutex.WaitOne(10000))
{
throw new Exception("Case with Id = " + CaseId + ": Could not obtain lock.");
}
try
{
if (SiAuto.Si.GetSession(CaseId.ToString()) == null) SiAuto.Si.AddSession(CaseId.ToString(), true);
dataContext = new ExrayDataContext(DBConnectionString);
if (SiAuto.Si.Level == Level.Debug)
dataContext.Log = new SmartInspectLinqToSqlAdapter(SiAuto.Si.GetSession(CaseId.ToString()))
{TitleLimit = 0, TitlePrefix = "WorkflowService: "};
Case objCase = Case.WorkflowService_ProcessCaseInt(dataContext, CaseId).SingleOrDefault();
if (objCase == null)
{
SiAuto.Si.GetSession(CaseId.ToString()).LogError("Case with Id = " + CaseId +
" does not exist.");
return;
}
if (objCase.Status == CaseEventStatusEnum.Closed || objCase.Status == CaseEventStatusEnum.Expired)
{
SiAuto.Si.GetSession(CaseId.ToString()).LogError("Case with Id = " + CaseId +
" is closed.");
return;
}

Event objCurrentEvent = null;
Event_EventType Event_EventType = GetCurrentEvent(dataContext, objCase.Id).FirstOrDefault();
if (Event_EventType != null)
{
objCurrentEvent = Event_EventType.Event;
objCurrentEvent.Type = Event_EventType.EventType;
objCurrentEvent.context = dataContext;
}
objCase.context = dataContext;
ProcessWorkflow(objCase, objCurrentEvent);
}
finally
{
if (dataContext != null) dataContext.Dispose();
if (LockMutex != null) LockMutex.ReleaseMutex();
if (SiAuto.Si.GetSession(CaseId.ToString()) != null)
{
SiAuto.Si.DeleteSession(SiAuto.Si.GetSession(CaseId.ToString()));
}
}
}
catch (Exception e)
{
HandleException(null, e);
throw;
}
}

StanislavK · Post by **StanislavK** » Thu 22 Sep 2011 16:54

As far as I can understand, this function merely performs the following:
- creates a DataContext instance;
- performs a query;
- disposes the DataContext object.

Provided that other data access functions of your application work in the same way (please specify if they do), the difference between this and other functions should be the query that is executed. Please describe it and the objects it works with, so that we are able to analyze the issue in more details.

crazypit · Post by **crazypit** » Tue 18 Oct 2011 13:22

Hello,

I switched to Direct connection to a node of my RAC for some days and everything worked as expected. But it's time i go back to my full RAC functionality and the problem persists. Do you want me to send you a crash dump with the relevant information? I don't know what else to do!

StanislavK · Post by **StanislavK** » Fri 21 Oct 2011 13:57

We have prepared you a test build, please try installing it and tell us if the problem persists in this case.

I will send you the details about downloading this build by email.

crazypit · Post by **crazypit** » Fri 21 Oct 2011 14:25

Im using version 6.30.202 of dotConnect or 2.50.30 of LinqConnect. Can i have a test build of one of this? I really dont want to get into possible breakdowns due to the newer minor version...

StanislavK · Post by **StanislavK** » Wed 26 Oct 2011 16:55

Sorry for the delay. We are working on the test build based on LinqConnect 2.50.30. We will inform you when it is available.

StanislavK · Post by **StanislavK** » Thu 27 Oct 2011 14:49

We've prepared the test build of LinqConnect 2.50.30. I will send you the download details in a letter, please check that it is not blocked by your mail filter.