16 ms·

Also remember that some ways are impossible to tell apart. 12/10/2008, for example, is 12th October for us in the UK, but 10th December for Americans. There's s

Also remember that some ways are impossible to tell apart. 12/10/2008, for example, is 12th October for us in the UK, but 10th December for Americans. There's simply no way to guess at which it is without some context or the user saying which. What that shows is that while NLP can make systems seem quite intelligent, it's really not a replacement for the user putting some thought into it too. Forms work nicely as they can force input into certain formats. I like NLP, so I always think systems could at least meet a human half way on this. However, even if we were talking face to face, there's some social expectation you'll try to make some sense (i.e. speak in English). You wouldn't just speak Spanish to me if I hadn't indicated I could speak Spanish. AI-like systems are meant to be intelligent, not psychic -- a point often forgotten by some people. A dialogue system would help uncertainties since it could ask for clarification (which, frankly, is all we humans do if something's not clear when talking to each other), but if you're talking over email, you really want to get it right the first time as it's not the fastest system. Again, off the top of my head: I'd say you could make it clear that the subject of the email has to be one of "reminder", "followup" or "todo"... then do some aggressive normalisation on the subject lines that come in (grab a Porter Stemmer or even just take the first 3/4 letters maybe) then force to upper or lower and then check against a massive list of synonyms. Then you simply have to bounce with a help notice if someone really still managed to enter something stupid. Examples: Subject: Reminder -> REMINDER -> REM -> matches rule for "REM/EVE/HOL" (Reminder, event, holiday) Subject: event -> EVENT -> EVE -> matches rule for "REM/EVE/HOL" Subject: follow-up -> FOLLOW-UP -> FOL -> matches rule for "FOL/REV" (follow-up, review) That sort of idea. It accounts for RaNDom CapITAlisation, uncertain hyphen-usage and plurals. Make those match rules long enough and you might get error rate pretty low indeed. I know you've probably already gone over this, but does it really need to be email? Is that the best interface? Is it worth taking a step back (going outside the box, as they say) and working out the most efficient method?

fendale 18y ago
There is some good advice coming up in this thread. A few people are suggesting email is not the correct medium, so I need to explain more what it is I am trying to achieve. My app is sort of like 'I Want Sandy' (I started working on this before I even heard about I Want Sandy, as I only came to know of it now that its about to be shut down). Basically, I want to be able to take an email sent to me, and forward it to a drop box. I can either just send the email in as is (received emails have certain information extracted from them and displayed on a dash board), or I can send it in with a command on it, or several commands. So I can fire in an email saying followup in 2 days/on Dec 12th etc, and in two days time, the app will send me an email along with the original email attached. If I add a command as TODO, then it will add it to the TODO list attached to that drop box etc. There will of course be a non-ambiguous web front end to all this to manage all the items, and add followups, todo's etc without an email, but my goal is to be able to add items from your inbox.
- AvengerPenguin 18y ago
  Ah, that's quite a neat idea really. I do think you're going to have to stick to a fixed syntax here though. People are already familiar with the standards of "Re: and Fw:" on the Internet, so it's not that far-fetched for people to learn "Todo:", "Followup:" and "Remind:". My overall suggestion would be to go with that (you could even tell users to press the "forward" button then replace "fw" with one of three commands). Then you can go with my suggestion of normalising caps, punctuation and even stemming to the first few letters to as to make it as forgiving as possible. e.g., FOL would even match a typo like Fololw-up ;-) That way, it could expect all subject lines in the form: Command: Text @ Date with date being optional perhaps. A standard regex or even just a split() would be fine here. If there's an error at any point, a standard help bounce would be good (think the --help text you get when you misuse a UNIX command). I really don't think it's unreasonable to expect users to get the command bit right, so the problem really is reduced to getting the date bit parsed correctly for which I've seen two suggestions here already. p.s. One possible future thing could be to make extensions for things like Thunderbird that actually gives a "Todo", "Remind" and "Follow-up" buttons alongside the "Reply" and "Forward" buttons ;-)
  - fendale 18y ago
    p.s. One possible future thing could be to make extensions for things like Thunderbird that actually gives a "Todo", "Remind" and "Follow-up" buttons alongside the "Reply" and "Forward" buttons ;-) That would be very much in the future! I actually have a a prototype working at the moment, but its for my eyes only for a while yet until I can iron a few things out! Good tip on the stemming - I will certainly give that a go.
- msbmsb 18y ago
  Note: I work in the NLP field. I'll echo what others have been suggesting, that in this case a restricted grammar with a thesaurus for synonyms and making use of some extant date libraries might be your best option here. I suggest this because your trigger input is purpose-driven. If you were going to be extracting this type of information from the middle of a email or document content, then I would be making some NLP(-ish) suggestions here. However, since you say that the commands to your application will be input directly by the user for that specific purpose, it will be much more user-friendly and easier (with more user confidence) if you have a pre-defined simple grammar with a range of available options and command synonyms. That way, since the user is only typing for your application to understand, they know what they can/should type for the expected results. You can allow for a little variety, a little variation in date descriptions, but it should be constrained to a point. The comments about auto-complete suggestions would work very well in reminding the user what types of commands are available. Have a list of commands (TODO, FOLLOWUP, NAGME, etc) and then when the user selects one of those commands, fill in the other parts of the syntax with placeholders (FOLLOWUP: [in X days|on DATE],...) perhaps. I'd say that you could add a little intelligence to it by picking up on certain keys from the email to suggest commands, i.e. dates in the email could be options for the date part of the command, and other more complex (and less-accurate) extractions. But a simpler solution should work better in this case than anything with a more NLP focus, in my opinion.

fendale 18y ago

There is some good advice coming up in this thread. A few people are suggesting email is not the correct medium, so I need to explain more what it is I am trying to achieve. My app is sort of like 'I Want Sandy' (I started working on this before I even heard about I Want Sandy, as I only came to know of it now that its about to be shut down). Basically, I want to be able to take an email sent to me, and forward it to a drop box. I can either just send the email in as is (received emails have certain information extracted from them and displayed on a dash board), or I can send it in with a command on it, or several commands. So I can fire in an email saying followup in 2 days/on Dec 12th etc, and in two days time, the app will send me an email along with the original email attached. If I add a command as TODO, then it will add it to the TODO list attached to that drop box etc. There will of course be a non-ambiguous web front end to all this to manage all the items, and add followups, todo's etc without an email, but my goal is to be able to add items from your inbox.

AvengerPenguin 18y ago
Ah, that's quite a neat idea really. I do think you're going to have to stick to a fixed syntax here though. People are already familiar with the standards of "Re: and Fw:" on the Internet, so it's not that far-fetched for people to learn "Todo:", "Followup:" and "Remind:". My overall suggestion would be to go with that (you could even tell users to press the "forward" button then replace "fw" with one of three commands). Then you can go with my suggestion of normalising caps, punctuation and even stemming to the first few letters to as to make it as forgiving as possible. e.g., FOL would even match a typo like Fololw-up ;-) That way, it could expect all subject lines in the form: Command: Text @ Date with date being optional perhaps. A standard regex or even just a split() would be fine here. If there's an error at any point, a standard help bounce would be good (think the --help text you get when you misuse a UNIX command). I really don't think it's unreasonable to expect users to get the command bit right, so the problem really is reduced to getting the date bit parsed correctly for which I've seen two suggestions here already. p.s. One possible future thing could be to make extensions for things like Thunderbird that actually gives a "Todo", "Remind" and "Follow-up" buttons alongside the "Reply" and "Forward" buttons ;-)
- fendale 18y ago
  p.s. One possible future thing could be to make extensions for things like Thunderbird that actually gives a "Todo", "Remind" and "Follow-up" buttons alongside the "Reply" and "Forward" buttons ;-) That would be very much in the future! I actually have a a prototype working at the moment, but its for my eyes only for a while yet until I can iron a few things out! Good tip on the stemming - I will certainly give that a go.
msbmsb 18y ago
Note: I work in the NLP field. I'll echo what others have been suggesting, that in this case a restricted grammar with a thesaurus for synonyms and making use of some extant date libraries might be your best option here. I suggest this because your trigger input is purpose-driven. If you were going to be extracting this type of information from the middle of a email or document content, then I would be making some NLP(-ish) suggestions here. However, since you say that the commands to your application will be input directly by the user for that specific purpose, it will be much more user-friendly and easier (with more user confidence) if you have a pre-defined simple grammar with a range of available options and command synonyms. That way, since the user is only typing for your application to understand, they know what they can/should type for the expected results. You can allow for a little variety, a little variation in date descriptions, but it should be constrained to a point. The comments about auto-complete suggestions would work very well in reminding the user what types of commands are available. Have a list of commands (TODO, FOLLOWUP, NAGME, etc) and then when the user selects one of those commands, fill in the other parts of the syntax with placeholders (FOLLOWUP: [in X days|on DATE],...) perhaps. I'd say that you could add a little intelligence to it by picking up on certain keys from the email to suggest commands, i.e. dates in the email could be options for the date part of the command, and other more complex (and less-accurate) extractions. But a simpler solution should work better in this case than anything with a more NLP focus, in my opinion.