4 ms·

I think you pointed out the main issue with arrays. If I follow your reasoning and suppose we will not want to know who is tagging what and when because we beli

by gbog 13y ago

I think you pointed out the main issue with arrays. If I follow your reasoning and suppose we will not want to know who is tagging what and when because we believe we just want to know the bare list of tags attached to an item. But then the app complexifies, we need more data, and we would like to have ansers to questions such as "is tagging different in the morning or in the evening?", "are there new trends in tags or new tags since last year?", or we even simply want to remove the tags created by some users because we know they try to cheat the system or spam it. Then with your "simple" solution I think we are stuck and will need to migrate, which is a pain. Moreover, all past data is unsuable. If, on the opposite, if we just do a "who tags what when" table, we are on much safer ground. We get a scalable event table that can be indexed (and sharded) by timestamp. We can undo any operation (which should be required for any action but email sending). The only drawbacks are 1) we need to aggregate and cache this data to have a list of tags for each item, and 2) we might have stored some data we might not need in the future. I always feel the saved data granularity should be one level finer than the data granularity used in the business model.

saurik 13y ago
> we even simply want to remove the tags created by some users because we know they try to cheat the system or spam it I'm not certain we are even talking about the same use case for "tags" here. If different users can tag the same post you start having to ask other interesting questions regarding permissions and ownership. The model I'm looking at is more blogger or delicious: the posts by users are distinct, and are owned on data records that are owned by those users. In the delicious case, we are all tagging the same URL eventually, but each user is really submitting a post for a URL with its own tag cloud. You cannot add tags to my post: you have to make your own post that references the same URL and put tags on that. The semantics of groups of people tagging things sounds awkward for arrays (and the one system I've built that was similar to that, where tags actually have ownership and permissions on their use even, did not go down this route). ===== (I wrote the following before noticing the incompatibility of your "a user somehow tagged something" case with my mental model of what we meant by tags; I still think it might be valuable, though.) I think that you are focussing on a narrow use of tags where all they can do is "list entries tagged by"; if that's all you are doing, then go ahead and use the repation. However, please understand that the app didn't "complexify" as you added those data items, and the data model didn't somehow improve to handle them: you lost the ability to do queries of the form "find documents similar to this one" without doing heroics. This is a trivial single select order by operator query with a cloud, and this is nearly impossible to represent using a separate relation. Now, think about the new queries you just wanted to perform: they were on properties not of present-tense taggings, but of past-tense ones. If someone adds a tag in the morning, and later deletes it, your business intelligence query still cares that they added that tag in the morning. This means you can't ever delete records, and end up adding "deleted" keys. This, in turn, makes your unique indexes require predicates, as all of your queries. It also means your "hot" user data includes "dead" information it has to sift through. Again, you're going to try to tell me this is a "micro optimization", but I'm going to say the exact opposite: for one ninche use case that end users don't even interact with, you made your app harder to implement (as the relation ways of accessing the tags is more awkward to code), less functional and slower (as now some queries are impossible at worst, often now unindexable when possible, and even at the best being bogged down by skipping data that serves a different purpose). This is the same argument against micro-optimization. Instead, I urge you to consider business intelligence issues as fundamentally different than your functionality issues. The way to solve this problem is to store events I how tags are use in a separate database, possibly a totally different kind of database, that is setup to store historical information of every tagging event that ever occurred (or, even, starts compressing this information into time windows). There is a reason why BI-specific database solutions exist, or why Amazon offers both red shift and DynamoDB: they are just different things.

by gbog 13y ago

saurik 13y ago

> we even simply want to remove the tags created by some users because we know they try to cheat the system or spam it I'm not certain we are even talking about the same use case for "tags" here. If different users can tag the same post you start having to ask other interesting questions regarding permissions and ownership. The model I'm looking at is more blogger or delicious: the posts by users are distinct, and are owned on data records that are owned by those users. In the delicious case, we are all tagging the same URL eventually, but each user is really submitting a post for a URL with its own tag cloud. You cannot add tags to my post: you have to make your own post that references the same URL and put tags on that. The semantics of groups of people tagging things sounds awkward for arrays (and the one system I've built that was similar to that, where tags actually have ownership and permissions on their use even, did not go down this route). ===== (I wrote the following before noticing the incompatibility of your "a user somehow tagged something" case with my mental model of what we meant by tags; I still think it might be valuable, though.) I think that you are focussing on a narrow use of tags where all they can do is "list entries tagged by"; if that's all you are doing, then go ahead and use the repation. However, please understand that the app didn't "complexify" as you added those data items, and the data model didn't somehow improve to handle them: you lost the ability to do queries of the form "find documents similar to this one" without doing heroics. This is a trivial single select order by operator query with a cloud, and this is nearly impossible to represent using a separate relation. Now, think about the new queries you just wanted to perform: they were on properties not of present-tense taggings, but of past-tense ones. If someone adds a tag in the morning, and later deletes it, your business intelligence query still cares that they added that tag in the morning. This means you can't ever delete records, and end up adding "deleted" keys. This, in turn, makes your unique indexes require predicates, as all of your queries. It also means your "hot" user data includes "dead" information it has to sift through. Again, you're going to try to tell me this is a "micro optimization", but I'm going to say the exact opposite: for one ninche use case that end users don't even interact with, you made your app harder to implement (as the relation ways of accessing the tags is more awkward to code), less functional and slower (as now some queries are impossible at worst, often now unindexable when possible, and even at the best being bogged down by skipping data that serves a different purpose). This is the same argument against micro-optimization. Instead, I urge you to consider business intelligence issues as fundamentally different than your functionality issues. The way to solve this problem is to store events I how tags are use in a separate database, possibly a totally different kind of database, that is setup to store historical information of every tagging event that ever occurred (or, even, starts compressing this information into time windows). There is a reason why BI-specific database solutions exist, or why Amazon offers both red shift and DynamoDB: they are just different things.