How are you "refreshing"? If you know a value has changed, you can Refresh the context cached value using ServerWins or ClientWins :
// Resolve the concurrency conflict by refreshing the
// object context before re-saving changes.
However, the real question comes down to when and why this would happen in the first place, which is related to the lifetime of the context. Perhaps you could be over-using the same context? Remember that SQL Server and etc have their own caching mechanism, and thus Entity Framework isn't the only fish in the pond of items trying to do some caching.
From, the following link we gather some tips, which if you were following, you likely wouldn't run into this problem. http://msdn.microsoft.com/en-us/data/jj729737.aspx
- As you load more objects, memory consumption of the context may increase rapidly
- Chances of running into concurrency related issues increases with expanded lifetime
- When working with Web Applications, use a context instance per request
- When working with Windows Forms, use a context per form