8 ms·

that's ... literally what people do with redis and memcache, you just wrote a slower memcache with fewer features. that's on one server and the data is all in

by zemo 7y ago

that's ... literally what people do with redis and memcache, you just wrote a slower memcache with fewer features. that's on one server and the data is all in process memory?

throwmeawy 7y ago
GP has shown a long-term tendency to create a naive implementation, in VB.NET even, and then get surly about folks who refuse to acknowledge its sufficiency. No good can come from engaging / trying to educate. N.b. built-in .NET object serialization is a slow, wasteful, brittle solution, but it could be okay if one doesn't mind the performance or portability issues. Heck why not use XmlSerializer then?
- graycat 7y ago
  What's wrong with VB.NET? As far as I can tell, it's as good as anything else to get to the .NET Framework. I like it because it has more traditional syntax I believe is easier to teach, learn, read, write, and debug than C#. With VB.Net, what am I giving up? I'm asking about "sufficiency": It all seems dirt simple to me, where some simple code really is sufficient. So, where is the simple approach not powerful enough? Where is Redis needed and something simple not good enough? Where does my simple solution fall short? It appears that Redis is now really important: What does it have that is so important? Is object instance serialization really slower than, all things considered, it has to be? Or if we accept (1) we are going to have instances of VB.Net style classes and (2) send the data via TCP/IP, that is, basically via byte arrays, then where can we do better than just serializing the class instances to byte arrays? If we work with just XML text, then we don't really have the advantages of performance of VB.Net instances of classes, and we're back to interpretive data structures instead of compiled data structures. I can think of some just blindingly fast approaches in PL/I because the PL/I data structures (1) are nearly as useful as class instances and (2) occupy essentially sequential storage -- so, could just copy storage from a starting address to an ending address and send the result as the byte array -- reverse that process at the receiver. Could do much the same with C structures, much less general than PL/I structures, if avoided most uses of pointers. This is my first Web site, first use of collection classes, first use of a session state store -- I'm trying to learn what I'm missing out on.
  - zemo 7y ago
    > What's wrong with VB.NET? nobody in the startup space is using it, so it'll be hard to hire for people that have the skills. There's a feedback loop there. Because no other companies are using it, people that don't have the skills also don't want to spend their time learning it, because the investment won't have a good pay off. I improved my resume and got more job offers by -removing- VB.NET ... nine years ago. > It appears that Redis is now really important: What does it have that is so important? It's really fast, really stable, and really well-understood by a lot of people. You can hire someone that already knows how to query it, how to operate it, how to fix it when it breaks. A lot of tools exist to work with it that you would have to write yourself if you rolled your own solution. When a bug is found, it's reported to the maintainers, and then fixed for everybody. > If we work with just XML text XML is pretty dead as a data interchange format. It's incredibly space-inefficient. Most people are using one of json, msgpack, thrift, or protocol buffers. (I'm personally a fan of protocol buffers; you write a schema and use the schema to generate the serialization and deserialization code, and the messages are efficiently-packed binary so they're pretty good on space efficiency.) There's a few outliers; mongodb uses bson because it predates the publication of msgpack but it's a very similar format. Writing the in-memory representation of a value and then reading it back on another machine and mmaping it and operating on it directly is a rare approach because it's hard to do it correctly while supporting changes to the types. Eventually you update an application, and an old record written by version 1 of the application has to be readable by version 2 of the application. It's not a completely dead strategy though, it's what cap'n proto does. seriously though I'm willing to bet there is almost nobody on Hacker News that understands any of your PL/I comparisons. You keep bringing up PL/I as if you assume everyone knows PL/I. In ten years as a professional programmer you are literally the only person I have ever heard bring up PL/I in a discussion that's not about history.
    - graycat 7y ago
      I have a box from IBM with a version of PL/I that can run on Windows. I did run it years ago on Windows XP. I no longer run or want to run PL/I. I don't expect anyone to use it. I bring up PL/I here on HN occasionally just as an example and lesson in programming language design. It's got some good features now lost, and for people interested in programming language design maybe some of the features could be brought back. Since there has been a complaint on this thread that .Net style object instance de/serialization is slow and otherwise not so good, part of the problem is that such an object instance is not all by itself in some range of sequential memory addresses and, instead, can be scattered all over. So, de/serialization needs implementation by the language developers. So, HAVING the desired data in sequential memory could have an advantage, e.g., in speed, that is, just copy the contents of memory in the block of addresses -- tough to be faster than that (ah, do it in assembler and use some string copy instruction assuming that X86 has one of those!). Could do similar things for the simpler versions of C structs. But in cautious shops, running code to get to memory addresses could be frowned on! A broad point about PL/I is that its structures are nearly as useful as the .Net style classes but with much faster and simpler addressing. In programming language design, that's a little point to keep in mind. I wrote enough C code to see how nearly all that syntax worked. So, maybe someday I'll write some C# code to be sure I know how its version of the C idiosyncratic syntax works! Since I'm a sole, solo founder, floor sweeper to CEO, of my own startup, I get to concentrate on the tools my startup needs instead of tools someone else's startup uses! The key-value store discussion here is good: I'm concluding that my needs are simple and my simple code is enough for those needs, that from the experience of others here in time I should want more functionality, and maybe should convert over to Redis. By then I might be hiring people to do such things. I'm not sure that having my code in VB.NET is a serious dead end: As I understand, C# can do what VB.NET does with just a different flavor of syntactic sugar, there's a translator from Microsoft to go from VB.NET to C#, and the actual semantics are so close that for the corresponding language features (some of the more recent features in C# may be missing from VB) it is fine to call one of the languages from the other. That is, as I understand the situation, the real work is to get to the .Net Framenwork and the CLR (common language runtime) and VB, C#, F#, etc. are all essentially equivalent for that work. So, for differences we're talking some functionality and otherwise syntactic sugar. My current work is more nonsense: Last night I started a run of Robocopy to do an incremental backup of one of my hard disk partitions. This morning Robocopy had completed without errors, but what it copied just as an incremental copy was about 80% of the whole disk partition when I've had only minor activity on that partition since the last full backup and turning off all the archive bits (archive bit OFF means that the file has been backup up since it was created or last changed and ON means that the file was created or changed since the last time the archive bit was set OFF). So, somehow that disk partition got a LOT of archive bits turned on. Open Object Rexx has a cute function can call to get a really nice description of everything in a file system subtree, one file/directory per line, with archive, hidden, system, read only bits, time-date stamp, size in bytes, full tree name. So, I compared two of these files and found that after the last full backup I really did set all the archive bits off and for the incremental a lot of REALLY old files, going back years, still had their correct time-date stamps, had NOT changed, but DID have their archive bits set ON. Bummer. Something has been walking around in that tree setting archive bits ON for NO good reason. Really big bummer. So, to explain, the partition drive letter is K:. There are some files in the root of K:, but mostly the files are in K:\data01, ... K:\data05 and K:prog01, ..., K:\prog05. Right, separation of data and programs! So, looking, all the archive bits set ON for no good reason were in just K:\data05. Hmm. What could have happened? Well, this computer is one I built from parts, running Windows 7 64 bit Professional SP1, but I also have an HP laptop running Windows 10 Home Edition with, of course, all the latest updates whether I wanted them or not. And a week or so ago, I got okay with net share net use etc. to set up file sharing between the two computers. Well, I gave the Windows 10 system access to the Windows 7 directory, right, you guessed it, K:\data05 That about has to be the cause. Somehow letting Windows 10 share the Windows 7 directory K:\data05 had the Windows 10 system walking around all over that directory, including several directory levels deep, and to some files years old, and setting archive bits ON. Bummer. So, no more such sharing! I'll create a directory, say, HP_laptop_share, in the root of a partition and let Windows 10 share it. I'll just put temporary stuff there. And unless there are more bugs, Windows 10 will be limited to making a mess only in that directory HP_laptop_share. So, with the HP OFF the Windows 7 system, I'll do a full backup, set the archive bits off, and keep Windows 10 the heck OUT of there! Lost half a day with this. I call it mud wrestling. I has NOTHING to do specifically with my startup. ALL the work unique to my startup has been fast, fun, and easy for me -- the difficulty has been literally YEARS of mud wrestling, all for no good reason. What do others do about such mud wrestling?
graycat 7y ago
Thanks. I'd heard of Redis -- it looked like overkill and more work just to understand than what I wrote. I'd never looked at memcache. With your mention of these two, I did a little Google search. Google has some of their little nutshell relevant descriptions right away and a also a link to an informative article: https://www.linkedin.com/pulse/memcached-vs-redis-which-one-pick-ranjeet-vimal https://www.linkedin.com/pulse/memcached-vs-redis-which-one-... This article seems to say that now Redis has the memcache functionality built-in implying that never need to use both. The article does seem to claim that memcache is multi-threaded and Redis, single. I've been hoping I could get by with single threaded and the TCP/IP FIFO queue -- IIRC .Net has a multi-threaded version of the collection class. Or, maybe if want multi-threaded, better just to run that many instances of the software, each single threaded, and f'get about the overhead for the multi-threaded collection classes? Again it looks like memcache and Redis are more complicated just to use than what I wrote. "You just wrote a slower memcache with fewer features." Well, well almost anything will have more features than I included! What I wrote should be plenty fast internally if the .Net collection classes I used are from AVL trees, Red-Black trees, or something similar. Then what I wrote would be slower because (i) get to it via TCP/IP sockets instead of some addressing more direct that would have to be on the same machine. But I didn't really want my session state store to have to be on the same machine as the Web server software that is using it. So, that essentially implies that a Web server has to get to the session state key-value store via TCP/IP sockets and stands to be about as fast as anything else with such socket access. Let's see: For some much heavier loads with lots of Web servers and lots of session state store key-value reads/writes, startup 20 such session state stores and have each key start with a number 1-20 to specify which key-value store. Then do a crude load leveling -- for a new key-value pair, send that to 1 plus the number of the last number used modulo 20, that is, round-robin. So, that would be my approach to something scalable, distributed, IIRC, sharding. Yes, could do more where could have an instance of the session state store start a shutdown be accepting no new key-value pairs and then after the time out value with all its pairs deleted send a message somewhere saying it is shutting down. A Web server trying to sent a new pair to such a shutting down server would receive back a going out of business message and not bother that server again until it has received another message. Or when intending a shutdown of a session state server, just send a notification to each Web server that the particular session state server was shutting down. So, could have some system management features for better continual operation. Maybe Redis has all this already and the details of such features are what I don't want to take time to read just now! Sounds like if my startup is successful, I'll be converting over to Redis!

by zemo 7y ago

that's ... literally what people do with redis and memcache, you just wrote a slower memcache with fewer features. that's on one server and the data is all in process memory?

throwmeawy 7y ago

GP has shown a long-term tendency to create a naive implementation, in VB.NET even, and then get surly about folks who refuse to acknowledge its sufficiency. No good can come from engaging / trying to educate. N.b. built-in .NET object serialization is a slow, wasteful, brittle solution, but it could be okay if one doesn't mind the performance or portability issues. Heck why not use XmlSerializer then?

graycat 7y ago
What's wrong with VB.NET? As far as I can tell, it's as good as anything else to get to the .NET Framework. I like it because it has more traditional syntax I believe is easier to teach, learn, read, write, and debug than C#. With VB.Net, what am I giving up? I'm asking about "sufficiency": It all seems dirt simple to me, where some simple code really is sufficient. So, where is the simple approach not powerful enough? Where is Redis needed and something simple not good enough? Where does my simple solution fall short? It appears that Redis is now really important: What does it have that is so important? Is object instance serialization really slower than, all things considered, it has to be? Or if we accept (1) we are going to have instances of VB.Net style classes and (2) send the data via TCP/IP, that is, basically via byte arrays, then where can we do better than just serializing the class instances to byte arrays? If we work with just XML text, then we don't really have the advantages of performance of VB.Net instances of classes, and we're back to interpretive data structures instead of compiled data structures. I can think of some just blindingly fast approaches in PL/I because the PL/I data structures (1) are nearly as useful as class instances and (2) occupy essentially sequential storage -- so, could just copy storage from a starting address to an ending address and send the result as the byte array -- reverse that process at the receiver. Could do much the same with C structures, much less general than PL/I structures, if avoided most uses of pointers. This is my first Web site, first use of collection classes, first use of a session state store -- I'm trying to learn what I'm missing out on.
- zemo 7y ago
  > What's wrong with VB.NET? nobody in the startup space is using it, so it'll be hard to hire for people that have the skills. There's a feedback loop there. Because no other companies are using it, people that don't have the skills also don't want to spend their time learning it, because the investment won't have a good pay off. I improved my resume and got more job offers by -removing- VB.NET ... nine years ago. > It appears that Redis is now really important: What does it have that is so important? It's really fast, really stable, and really well-understood by a lot of people. You can hire someone that already knows how to query it, how to operate it, how to fix it when it breaks. A lot of tools exist to work with it that you would have to write yourself if you rolled your own solution. When a bug is found, it's reported to the maintainers, and then fixed for everybody. > If we work with just XML text XML is pretty dead as a data interchange format. It's incredibly space-inefficient. Most people are using one of json, msgpack, thrift, or protocol buffers. (I'm personally a fan of protocol buffers; you write a schema and use the schema to generate the serialization and deserialization code, and the messages are efficiently-packed binary so they're pretty good on space efficiency.) There's a few outliers; mongodb uses bson because it predates the publication of msgpack but it's a very similar format. Writing the in-memory representation of a value and then reading it back on another machine and mmaping it and operating on it directly is a rare approach because it's hard to do it correctly while supporting changes to the types. Eventually you update an application, and an old record written by version 1 of the application has to be readable by version 2 of the application. It's not a completely dead strategy though, it's what cap'n proto does. seriously though I'm willing to bet there is almost nobody on Hacker News that understands any of your PL/I comparisons. You keep bringing up PL/I as if you assume everyone knows PL/I. In ten years as a professional programmer you are literally the only person I have ever heard bring up PL/I in a discussion that's not about history.
  - graycat 7y ago
    I have a box from IBM with a version of PL/I that can run on Windows. I did run it years ago on Windows XP. I no longer run or want to run PL/I. I don't expect anyone to use it. I bring up PL/I here on HN occasionally just as an example and lesson in programming language design. It's got some good features now lost, and for people interested in programming language design maybe some of the features could be brought back. Since there has been a complaint on this thread that .Net style object instance de/serialization is slow and otherwise not so good, part of the problem is that such an object instance is not all by itself in some range of sequential memory addresses and, instead, can be scattered all over. So, de/serialization needs implementation by the language developers. So, HAVING the desired data in sequential memory could have an advantage, e.g., in speed, that is, just copy the contents of memory in the block of addresses -- tough to be faster than that (ah, do it in assembler and use some string copy instruction assuming that X86 has one of those!). Could do similar things for the simpler versions of C structs. But in cautious shops, running code to get to memory addresses could be frowned on! A broad point about PL/I is that its structures are nearly as useful as the .Net style classes but with much faster and simpler addressing. In programming language design, that's a little point to keep in mind. I wrote enough C code to see how nearly all that syntax worked. So, maybe someday I'll write some C# code to be sure I know how its version of the C idiosyncratic syntax works! Since I'm a sole, solo founder, floor sweeper to CEO, of my own startup, I get to concentrate on the tools my startup needs instead of tools someone else's startup uses! The key-value store discussion here is good: I'm concluding that my needs are simple and my simple code is enough for those needs, that from the experience of others here in time I should want more functionality, and maybe should convert over to Redis. By then I might be hiring people to do such things. I'm not sure that having my code in VB.NET is a serious dead end: As I understand, C# can do what VB.NET does with just a different flavor of syntactic sugar, there's a translator from Microsoft to go from VB.NET to C#, and the actual semantics are so close that for the corresponding language features (some of the more recent features in C# may be missing from VB) it is fine to call one of the languages from the other. That is, as I understand the situation, the real work is to get to the .Net Framenwork and the CLR (common language runtime) and VB, C#, F#, etc. are all essentially equivalent for that work. So, for differences we're talking some functionality and otherwise syntactic sugar. My current work is more nonsense: Last night I started a run of Robocopy to do an incremental backup of one of my hard disk partitions. This morning Robocopy had completed without errors, but what it copied just as an incremental copy was about 80% of the whole disk partition when I've had only minor activity on that partition since the last full backup and turning off all the archive bits (archive bit OFF means that the file has been backup up since it was created or last changed and ON means that the file was created or changed since the last time the archive bit was set OFF). So, somehow that disk partition got a LOT of archive bits turned on. Open Object Rexx has a cute function can call to get a really nice description of everything in a file system subtree, one file/directory per line, with archive, hidden, system, read only bits, time-date stamp, size in bytes, full tree name. So, I compared two of these files and found that after the last full backup I really did set all the archive bits off and for the incremental a lot of REALLY old files, going back years, still had their correct time-date stamps, had NOT changed, but DID have their archive bits set ON. Bummer. Something has been walking around in that tree setting archive bits ON for NO good reason. Really big bummer. So, to explain, the partition drive letter is K:. There are some files in the root of K:, but mostly the files are in K:\data01, ... K:\data05 and K:prog01, ..., K:\prog05. Right, separation of data and programs! So, looking, all the archive bits set ON for no good reason were in just K:\data05. Hmm. What could have happened? Well, this computer is one I built from parts, running Windows 7 64 bit Professional SP1, but I also have an HP laptop running Windows 10 Home Edition with, of course, all the latest updates whether I wanted them or not. And a week or so ago, I got okay with net share net use etc. to set up file sharing between the two computers. Well, I gave the Windows 10 system access to the Windows 7 directory, right, you guessed it, K:\data05 That about has to be the cause. Somehow letting Windows 10 share the Windows 7 directory K:\data05 had the Windows 10 system walking around all over that directory, including several directory levels deep, and to some files years old, and setting archive bits ON. Bummer. So, no more such sharing! I'll create a directory, say, HP_laptop_share, in the root of a partition and let Windows 10 share it. I'll just put temporary stuff there. And unless there are more bugs, Windows 10 will be limited to making a mess only in that directory HP_laptop_share. So, with the HP OFF the Windows 7 system, I'll do a full backup, set the archive bits off, and keep Windows 10 the heck OUT of there! Lost half a day with this. I call it mud wrestling. I has NOTHING to do specifically with my startup. ALL the work unique to my startup has been fast, fun, and easy for me -- the difficulty has been literally YEARS of mud wrestling, all for no good reason. What do others do about such mud wrestling?

graycat 7y ago

Thanks. I'd heard of Redis -- it looked like overkill and more work just to understand than what I wrote. I'd never looked at memcache. With your mention of these two, I did a little Google search. Google has some of their little nutshell relevant descriptions right away and a also a link to an informative article: https://www.linkedin.com/pulse/memcached-vs-redis-which-one-pick-ranjeet-vimal https://www.linkedin.com/pulse/memcached-vs-redis-which-one-... This article seems to say that now Redis has the memcache functionality built-in implying that never need to use both. The article does seem to claim that memcache is multi-threaded and Redis, single. I've been hoping I could get by with single threaded and the TCP/IP FIFO queue -- IIRC .Net has a multi-threaded version of the collection class. Or, maybe if want multi-threaded, better just to run that many instances of the software, each single threaded, and f'get about the overhead for the multi-threaded collection classes? Again it looks like memcache and Redis are more complicated just to use than what I wrote. "You just wrote a slower memcache with fewer features." Well, well almost anything will have more features than I included! What I wrote should be plenty fast internally if the .Net collection classes I used are from AVL trees, Red-Black trees, or something similar. Then what I wrote would be slower because (i) get to it via TCP/IP sockets instead of some addressing more direct that would have to be on the same machine. But I didn't really want my session state store to have to be on the same machine as the Web server software that is using it. So, that essentially implies that a Web server has to get to the session state key-value store via TCP/IP sockets and stands to be about as fast as anything else with such socket access. Let's see: For some much heavier loads with lots of Web servers and lots of session state store key-value reads/writes, startup 20 such session state stores and have each key start with a number 1-20 to specify which key-value store. Then do a crude load leveling -- for a new key-value pair, send that to 1 plus the number of the last number used modulo 20, that is, round-robin. So, that would be my approach to something scalable, distributed, IIRC, sharding. Yes, could do more where could have an instance of the session state store start a shutdown be accepting no new key-value pairs and then after the time out value with all its pairs deleted send a message somewhere saying it is shutting down. A Web server trying to sent a new pair to such a shutting down server would receive back a going out of business message and not bother that server again until it has received another message. Or when intending a shutdown of a session state server, just send a notification to each Web server that the particular session state server was shutting down. So, could have some system management features for better continual operation. Maybe Redis has all this already and the details of such features are what I don't want to take time to read just now! Sounds like if my startup is successful, I'll be converting over to Redis!