simplest key value store for xml data with best performance for .NET

+2 votes
asked Aug 27, 2010 by cetin-sert

I need a way to store large-numbers of variously sized XML data in a simple key/value store. So far I have tried two simple approaches:

  1. RAM: Store everything in a ConcurrentDictionary instance
    • Problem: limited resources at project launch, a single server with 8GBs of RAM!
    • Note: Windows AppFabric could be a solution here in the future if we get more capable servers.
  2. HDD: Store every value in its own file on the hard disk, with keys mapping to paths
    • Problem: latency
    • Question: how much of a difference could using an SSD make in the most optimistic scenario here?

Is there a ready-to-use database solution that could provide the middle-way, providing very fast read-access times while utilizing the HDD?

Write-access is only necessary for maintenance so any solution that specializes in least latency for read-only scenarios would work best. The data server should be communicable using WCF.

2 Answers

+2 votes
answered Aug 13, 2013 by spence

You might want to look at memcached if you are in .Net. It allows you start with in memory cache and move to redundant storage if required. It all depends on your data requirements and how often you are reading and writing the store.

As for a database, MSSQL provides XML type storage, so you could definately create a simple table with XML inside it, but it's not cheap and you did mention you wanted a NoSQL implementation. Note that if you need to search or sort your XML you may find having XPath and indexing available to be a bonus...

0 votes
answered Nov 29, 2018 by max-zerbini is a distribuited key/value store that works only in-memory. It's scalable and supports data replication and data expiration. The client library is based on WCF. It's available from Nuget.

Welcome to Q&A, where you can ask questions and receive answers from other members of the community.