5 ms·

Show HN: I Ported LevelDB to Universal Windows Platform

83 points by maxpert 11y ago

kozukumi 11y ago
Have you written about how you found the process? The good and bad things, etc. With UWP being so new and untested it is hard to find much information on porting an already established codebase to the platform. How is performance, etc?
- NicoJuicy 11y ago
  I'm not noticing any performance difference, porting isn't much of a hassle. I ported some layers of my DDD-webapplication ( huge codebase) to UWP to support Windows RT. So i could re-use my Api Client Connector. The biggest problem i ran into was with reflection. So i split up one of my nuget package to a UWP-supported DLL and an "extension" dll. Which contained the non UWP/PCL supported methods / functions. The UWP is used in my Api Connector and the extension dll is widely included into my "web application" + other apps ( import, export, wpf client, website, webapi ...). The other thing ( not really a problem) was that i dropped support for SilverLight since one of the Nuget DLL's i use, didn't support that. And i don't use SilverLight in the current situation, so the choice was easily made. Also, PCL requires async methods, so i had to change my Http Calls to support Async and use a different method to call the HTTP ( which really didn't take a long time). But i used a dirty trick to support Async ( eg. an extention method that extends uses my IApiInterface, to convert it to Async :) -- I needed it fast and didn't noticed any downsides till now. PS. UWP supported dll is called Portable Class Library ( PCL) if you want to look it up.
- maxpert 11y ago
  Hey thanks for your interest, I have been working on UWP for quite a while now; it has been both good and bad. My biggest gripe right now is no good DB support (SQLite requires an extension and that two is not enough; you have next to nothing ORMs and well EF7 is still RC). In today's world I can't image an App not finding database as a useful platform feature, Android/iOS have builtin DB support and they prove the fact how important DB support could be. I just don't understand what were people smoking when they made a decision to not include a DB framework in UWP. XAML framework has it's own bugs, caveats, and yes there is next to nothing material available to help you. They have improved a-lot with x:Bind but then you find bugs like this https://social.msdn.microsoft.com/Forums/en-US/c2bde144-47ae-4eb9-817f-1f434caa05f5/uwpcxaml-compiled-binding-xbind-cause-memory-leaks?forum=wpdevelop https://social.msdn.microsoft.com/Forums/en-US/c2bde144-47ae... over which you would spend a whole week and then say "AH FUCK IT; use Binding instead". Performance is good with .NET native but not as good as you would picture a native code to be. I have seen Java blowing my mind just with JIT. Personally I felt Microsoft has done good job of performance on PC since they have mastered the x86/x64, but ARM feels a little slow to me. I personally strongly feel (coming from WP 8.1 platform) that UWP is better but it feels like an RC; it has broken ends, missing good documentation (hell wrong documentation sometimes), and next to nothing opensource community support. One of my colleagues clearly said that if he has to choose to work on another UWP project he would choose to not work on it.
flohofwoe 11y ago
This nicely demonstrates the main design blunder of the Universal Windows Platform: introducing an incompatible dialect of C++ (called C++/CX, formerly known as C++/CLI, formerly known as Managed C++). I really wonder why Microsoft had to invent their own, incompatible C++ dialect when projects like emscripten can cross-compile perfectly standard C++ to Javascript, which is a much more exotic compile target than the .NET runtime.
- deleted 11y ago
  [deleted]
- skrebbel 11y ago
  Performance. An emscripten-like project could probably never generate IL that's as fast as the output of hand-ported C++. Or, at least, that must've been a sane opinion to hold back when managed c++ was first invented.
- pavlov 11y ago
  Confusingly, C++/CX is actually not for the .NET runtime, so it's not a successor to C++/CLI. Instead C++/CX targets the traditional COM platform which goes back decades and was kind of resurrected as the foundation of WinRT (and whatever it's called in Windows 10 -- UWP I guess? I have trouble following these rebrandings). As far as I've understood, the point of C++/CX is somewhat similar to Objective-C. You can still write raw COM code in plain C or C++ (just like you can use the Obj-C runtime from plain C if you really want to)... But C++/CX is there to make it more palatable.
  - nikbackm 11y ago
    You also don't need to use C++/CX in order to write UWP applications, there are several other bindings available [0]. There even exists a non-Microsoft made binding [1]. [0]https://en.wikipedia.org/wiki/Windows_Runtime#Programming_interfaces https://en.wikipedia.org/wiki/Windows_Runtime#Programming_in... [1] http://moderncpp.com/about/ http://moderncpp.com/about/
- notreallyright 11y ago
roryisok 11y ago
Any chance of a version for WinJS? The only javascript based database available to both Windows Phone and Windows Store apps is SQLite, and it's seriously buggy
- maxpert 11y ago
  Didn't thought about it but I think there exists a version already
- WorldMaker 11y ago
  You should be able to use this like any other WinRT component in WinJS. SQLite support should hopefully get better now that it has been added to the platform SDK. Also, have you tried PouchDB yet? I've gotten some good results from it.
  - maxpert 11y ago
    When was it added to Platform SDK? I am using VS 2015 Update 1.
    - WorldMaker 11y ago
      Looks like Platform SDK 10586: https://blogs.windows.com/buildingapps/2015/12/03/windows-10-build-10586-sdk-breakdown/ https://blogs.windows.com/buildingapps/2015/12/03/windows-10... Edit to add: Believe this is the same SDK included in VS 2015 Update 1.
Const-me 11y ago
But why? Windows already has ESENT. That embedded NoSQL database engine is developed and supported by Microsoft. Available since windows 2000, on the phones since 8.0. The functionality is much more. While you surely can use it as a key-value store, if you need more there’re secondary indices, column schema, multivalued columns, version columns, auto-increment columns, sparse indices, transactions…
- maxpert 11y ago
  Can you point me any tutorial or post that does a walk through on how to use ESENT on UWP applications? EDIT: I found ManagedESENT and yes there is a way to do it in UWP apps too but API seems like it's from stone-age example: Api.JetCreateInstance(out instance, "instance"); Api.JetSetSystemParameter(instance, JET_SESID.Nil, JET_param.CircularLog, 1, null); Api.JetInit(ref instance);
  - Const-me 11y ago
    ESENT is a low level API with C interface. It doesn’t matter on which platform you call those API. The differences are negligible. The main one — restriction on which parts of file system are writeable, on UWP you want to place your DB somewhere in ApplicationData.Current.LocalFolder As for a starting point, I can confirm sample code from there http://lunarfrog.com/blog/extensible-storage-engine http://lunarfrog.com/blog/extensible-storage-engine works OK with Windows 10 UWP, Visual Studio 2015 Update 1, and ManagedEsent 1.9.3.2 from NuGet. It’s very simple, but it works.
  - Const-me 11y ago
    As for the API, I agree that on UWP, it’s too low level. For desktop platforms, I’ve built my own ORM on top of that: https://esentserialize.codeplex.com/ https://esentserialize.codeplex.com/ However, I’m too lazy to port that on Windows Store unless I’m paid for that: the main problem isn’t ESENT, it’s the reflection that I use extensively for serializing/deserialising those records/fields. Edit: but if you only need key-value storage with both keys and values being blob of bytes, the complete implementation on top of Managed ESENT will be like four pages of C# code. The main complexity in ESENT isn’t DB initialization. For initialization, you copy-paste DB initialization code from PersistentDictionary or some other sample and forget about that (until much later when you'll be optimizing the DB performance). The hard parts are column schema, index schema, queries, schema upgrades, and also on servers it’s concurrency.

83 points by maxpert 11y ago

Have you written about how you found the process? The good and bad things, etc. With UWP being so new and untested it is hard to find much information on porting an already established codebase to the platform. How is performance, etc?

NicoJuicy 11y ago
I'm not noticing any performance difference, porting isn't much of a hassle. I ported some layers of my DDD-webapplication ( huge codebase) to UWP to support Windows RT. So i could re-use my Api Client Connector. The biggest problem i ran into was with reflection. So i split up one of my nuget package to a UWP-supported DLL and an "extension" dll. Which contained the non UWP/PCL supported methods / functions. The UWP is used in my Api Connector and the extension dll is widely included into my "web application" + other apps ( import, export, wpf client, website, webapi ...). The other thing ( not really a problem) was that i dropped support for SilverLight since one of the Nuget DLL's i use, didn't support that. And i don't use SilverLight in the current situation, so the choice was easily made. Also, PCL requires async methods, so i had to change my Http Calls to support Async and use a different method to call the HTTP ( which really didn't take a long time). But i used a dirty trick to support Async ( eg. an extention method that extends uses my IApiInterface, to convert it to Async :) -- I needed it fast and didn't noticed any downsides till now. PS. UWP supported dll is called Portable Class Library ( PCL) if you want to look it up.
maxpert 11y ago
Hey thanks for your interest, I have been working on UWP for quite a while now; it has been both good and bad. My biggest gripe right now is no good DB support (SQLite requires an extension and that two is not enough; you have next to nothing ORMs and well EF7 is still RC). In today's world I can't image an App not finding database as a useful platform feature, Android/iOS have builtin DB support and they prove the fact how important DB support could be. I just don't understand what were people smoking when they made a decision to not include a DB framework in UWP. XAML framework has it's own bugs, caveats, and yes there is next to nothing material available to help you. They have improved a-lot with x:Bind but then you find bugs like this https://social.msdn.microsoft.com/Forums/en-US/c2bde144-47ae-4eb9-817f-1f434caa05f5/uwpcxaml-compiled-binding-xbind-cause-memory-leaks?forum=wpdevelop https://social.msdn.microsoft.com/Forums/en-US/c2bde144-47ae... over which you would spend a whole week and then say "AH FUCK IT; use Binding instead". Performance is good with .NET native but not as good as you would picture a native code to be. I have seen Java blowing my mind just with JIT. Personally I felt Microsoft has done good job of performance on PC since they have mastered the x86/x64, but ARM feels a little slow to me. I personally strongly feel (coming from WP 8.1 platform) that UWP is better but it feels like an RC; it has broken ends, missing good documentation (hell wrong documentation sometimes), and next to nothing opensource community support. One of my colleagues clearly said that if he has to choose to work on another UWP project he would choose to not work on it.

flohofwoe 11y ago

This nicely demonstrates the main design blunder of the Universal Windows Platform: introducing an incompatible dialect of C++ (called C++/CX, formerly known as C++/CLI, formerly known as Managed C++). I really wonder why Microsoft had to invent their own, incompatible C++ dialect when projects like emscripten can cross-compile perfectly standard C++ to Javascript, which is a much more exotic compile target than the .NET runtime.

deleted 11y ago
[deleted]
skrebbel 11y ago
Performance. An emscripten-like project could probably never generate IL that's as fast as the output of hand-ported C++. Or, at least, that must've been a sane opinion to hold back when managed c++ was first invented.
pavlov 11y ago
Confusingly, C++/CX is actually not for the .NET runtime, so it's not a successor to C++/CLI. Instead C++/CX targets the traditional COM platform which goes back decades and was kind of resurrected as the foundation of WinRT (and whatever it's called in Windows 10 -- UWP I guess? I have trouble following these rebrandings). As far as I've understood, the point of C++/CX is somewhat similar to Objective-C. You can still write raw COM code in plain C or C++ (just like you can use the Obj-C runtime from plain C if you really want to)... But C++/CX is there to make it more palatable.
- nikbackm 11y ago
  You also don't need to use C++/CX in order to write UWP applications, there are several other bindings available [0]. There even exists a non-Microsoft made binding [1]. [0]https://en.wikipedia.org/wiki/Windows_Runtime#Programming_interfaces https://en.wikipedia.org/wiki/Windows_Runtime#Programming_in... [1] http://moderncpp.com/about/ http://moderncpp.com/about/
notreallyright 11y ago

roryisok 11y ago

Any chance of a version for WinJS? The only javascript based database available to both Windows Phone and Windows Store apps is SQLite, and it's seriously buggy

maxpert 11y ago
Didn't thought about it but I think there exists a version already
WorldMaker 11y ago
You should be able to use this like any other WinRT component in WinJS. SQLite support should hopefully get better now that it has been added to the platform SDK. Also, have you tried PouchDB yet? I've gotten some good results from it.
- maxpert 11y ago
  When was it added to Platform SDK? I am using VS 2015 Update 1.
  - WorldMaker 11y ago
    Looks like Platform SDK 10586: https://blogs.windows.com/buildingapps/2015/12/03/windows-10-build-10586-sdk-breakdown/ https://blogs.windows.com/buildingapps/2015/12/03/windows-10... Edit to add: Believe this is the same SDK included in VS 2015 Update 1.

Const-me 11y ago

But why? Windows already has ESENT. That embedded NoSQL database engine is developed and supported by Microsoft. Available since windows 2000, on the phones since 8.0. The functionality is much more. While you surely can use it as a key-value store, if you need more there’re secondary indices, column schema, multivalued columns, version columns, auto-increment columns, sparse indices, transactions…

maxpert 11y ago
Can you point me any tutorial or post that does a walk through on how to use ESENT on UWP applications? EDIT: I found ManagedESENT and yes there is a way to do it in UWP apps too but API seems like it's from stone-age example: Api.JetCreateInstance(out instance, "instance"); Api.JetSetSystemParameter(instance, JET_SESID.Nil, JET_param.CircularLog, 1, null); Api.JetInit(ref instance);
- Const-me 11y ago
  ESENT is a low level API with C interface. It doesn’t matter on which platform you call those API. The differences are negligible. The main one — restriction on which parts of file system are writeable, on UWP you want to place your DB somewhere in ApplicationData.Current.LocalFolder As for a starting point, I can confirm sample code from there http://lunarfrog.com/blog/extensible-storage-engine http://lunarfrog.com/blog/extensible-storage-engine works OK with Windows 10 UWP, Visual Studio 2015 Update 1, and ManagedEsent 1.9.3.2 from NuGet. It’s very simple, but it works.
- Const-me 11y ago
  As for the API, I agree that on UWP, it’s too low level. For desktop platforms, I’ve built my own ORM on top of that: https://esentserialize.codeplex.com/ https://esentserialize.codeplex.com/ However, I’m too lazy to port that on Windows Store unless I’m paid for that: the main problem isn’t ESENT, it’s the reflection that I use extensively for serializing/deserialising those records/fields. Edit: but if you only need key-value storage with both keys and values being blob of bytes, the complete implementation on top of Managed ESENT will be like four pages of C# code. The main complexity in ESENT isn’t DB initialization. For initialization, you copy-paste DB initialization code from PersistentDictionary or some other sample and forget about that (until much later when you'll be optimizing the DB performance). The hard parts are column schema, index schema, queries, schema upgrades, and also on servers it’s concurrency.