17 ms·
Building a Text Editor for a Digital-First Newsroom
- sshine 9y ago> If you’re like most people in America, you use a text editor nearly every day. Filter bubble detected.
- venning 9y agoThe author does not say "work in a text editor", only "use". Considering she's including text editors on mobile devices, which the majority of America will utilize today, that's not a particularly bold statement.
- optimuspaul 9y agoI suppose when I send a text I am using a text editor of sorts
- evrydayhustling 9y agoDisagree. Paragraph context and examples made it clear she is discussing a dedicated text editor, not something embedded in e.g. your SMS app. Can't imagine why parent is getting downvotes - it's an important point, and doesn't seek to invalidate the whole piece.
- venning 9y agoData Warning: 100 MB. Viewing the header illustration for more than a few seconds crashes my iPad Safari tab. Scrolling until it is off the viewport prevents this. EDIT: Dev Tools reports 74.8 MB transferred with an ad blocker. The illustration is a single 69.2 MB gif. EDIT: 101 MB in total when you scroll to the bottom. Three more gifs: 18.1 MB, 7.1 MB, 4.7 MB.
- vowelless 9y agoUnbelievable
- lambda 9y agoWhy are they gifs instead of mp4s? Yikes. Also, since they are screenshots without a lot of movement, they really should compress better than that even as gifs.
- slig 9y agoMaybe because they're using Medium, and Medium doesn't have that option. Why the biggest newspaper in America can't run their own hosted blog is a bigger question.
- SippinLean 9y ago>While it may seem strange for us, a publisher, to post on Medium rather than our own platform, we are here for a simple reason: the community. Medium is where so many of our people are, so much of the product, design and development community, so we wanted to be here too. https://open.nytimes.com/introducing-the-new-open-blog-23eba4463c59 https://open.nytimes.com/introducing-the-new-open-blog-23eba...
- Analemma_ 9y agoI'm sure they could run a hosted blog, it's probably a deliberate decision by their tech team to seem cool and hip by posting on Medium, because that's the in thing right now for developer blogs. (Note: I'm not dissing them, I know a very talented developer who works there and says he has a great team)
- slig 9y ago> because that's the in thing right now for developer blogs. Which is bizarre. The comments section of tech blogs are a great place to discuss and ask questions, and the support for comments on Medium is abismal, looks like they deliberately made it suck.
- skybrian 9y agoI think Hacker News does a better job of this, and so here we are.
- twothamendment 9y ago69 MB gif - isn't that called a video? I thought we had different formats for that.
- sli 9y agoWhat I find amazing is that it's that large despite only being two bits of color depth, and it's extremely short and low framerate. Where the hell did all that filesize come from?
- amelius 9y agoScreen-grabber?
- wjp3 9y agoCrypto currency mining! /joke
- benjaminl 9y agoLooks like they cut the size down some. I am seeing 10.7 MB for the GIF. There are several reasons for why the GIF is so large that all contribute to the size. The GIF's resolution is quite large for a GIF, at 1004px by 788px, this is larger then they display it on the page. And the GIF is pretty long at 336 frames. Since GIFs on like just about every video format except MJPEG has no interframe compression. So each frame only has to average 32 KB to make up that 10 MB (336 * 32.6KB = 10.7 MB). This is an example of how extremely good our modern video codecs are. [0] - https://ezgif.com/split/ezgif-4-240b758f26.gif https://ezgif.com/split/ezgif-4-240b758f26.gif
- tedmiston 9y agoInstapaper Text view reduces it to a slightly more reasonable 41 MB. https://www.instapaper.com/text?u=https%3A%2F%2Fopen.nytimes.com%2Fbuilding-a-text-editor-for-a-digital-first-newsroom-f1cb8367fc21 https://www.instapaper.com/text?u=https%3A%2F%2Fopen.nytimes... (This may require a free Instapaper account to view.)
- deleted 9y ago[deleted]
- flukus 9y agoUBlock brings it down to 204KB because the images are on a CDN. Being on a CDN is probably why the author is oblivious to the file sizes.
- dingo_bat 9y agoThere's something wrong with your iPad or safari itself. My 2 year old phone has no problems keeping up. It is incredibly stupid to have a 69 MB gif though.
- deleted 9y ago[deleted]
- chrisweekly 9y agoThat's kind of outrageous. Esp since the nytimes webdev team has generally done such a bang-up job w/ compelling UX.
- manigandham 9y agoCool, but it's yet another case of re-engineering the same thing. (edit: its not entirely new). Splitting editors into an abstracted structure for finer control over data and presentation is something that's been done for a decade now in various text editors. Medium and Basecamp started similar projects and there's Mobiledoc which is trying to standardize the format. It'd be nice if all this effort focused on a single project so we could all just get a decent editor instead of different versions with various states of functionality. FYI, my running list of editors: https://gist.github.com/manigandham/65543a0bc2bf7006a487 https://gist.github.com/manigandham/65543a0bc2bf7006a487
- zokier 9y agoConsidering that they have built it on top of ProseMirror makes it seem pretty reasonable approach in terms of "reinventing the wheel"
- lambda 9y agoThis is discussing one that you already had on your list; ProseMirror. In fact, you've had it on the list since 2015: https://gist.github.com/manigandham/65543a0bc2bf7006a487/429266dfa5a5686c6051a2cf0141e81b6792e5b8 https://gist.github.com/manigandham/65543a0bc2bf7006a487/429... This is talking about using ProseMirror to build some more features into their CMS, but it sounds like more of just a use of ProseMirror rather than a new editor entirely.
- deleted 9y ago[deleted]
- zawerf 9y agoIt might be helpful to update your list with some feature comparisons. For example: Quill vs (CKEditor and TinyMCE, Draft, ProseMirror, Trix): https://quilljs.com/guides/comparison-with-other-rich-text-editors/ https://quilljs.com/guides/comparison-with-other-rich-text-e... Slate vs (Draft, ProseMirror, Quill): https://github.com/ianstormtaylor/slate#why https://github.com/ianstormtaylor/slate#why I personally went with Slate because I needed first-class React support but not as low level as Draft.
- zellyn 9y agoAnyone know if Oak is (or will be) released as Open Source?
- lambda 9y agoProseMirror, the underlying editor they are using, is: https://prosemirror.net/ https://prosemirror.net/
- zellyn 9y agoYep. That was clear from the article.
- parvenu74 9y agoI suspect that NYT will contribute quite a bit back to ProseMirror out of enlightened self-interest but not give away the secret sauce that make Oak a competitive advantage for them. This is REAL freedom in software: without the ability to build a closed-source, for-profit business it won't succeed. I'll take MIT/BSD/Apache over something with a herpes license (GPL/AGPL) any day.
- geofft 9y agoCan we leave the Slashdot-quality trolling about licensing back in the previous millennium where it belongs? There are, of course, successful projects licensed under the GPL and AGPL (Linux comes to mind), many of which have for-profit businesses; there are plenty of people with different priorities about what "REAL freedom" is. And calling the licenses that match their priorities "herpes licenses" is unlikely to convince them.
- parvenu74 9y agoTrolling aside, the freedom allowed to not show all of your cards that MIT/MSD/Apache allows is better for business, and if the app is essential to their business then they will contribute fixes back upstream. An example is Juniper and their myriad contributions to FreeBSD. This allows Juniper to pull down upstream updates without it being a complete goat-rope to integrate with their own secret sauce code; if Juniper were using Linux instead of FreeBSD then anyone could demand full source code and sell knock-off solutions based on Juniper's work and the result would be less actual development of the technology (for lack of profit motive).
- jbob2000 9y agoWhew, what a gross way to over-engineer a text editor. If the NYTimes structured themselves like a dev team, they could use common sense tools to get much better results. Why is the journalist responsible for how it's presented? Wouldn't it make more sense to have some kind of product manager or designer handle this? And why is the journalist responsible for programming the look and feel of the their article? Why not have a front end dev do that?
- jmbwell 9y agoBusiness choices, perhaps, aimed at reducing costs by reducing editors and moving more responsibility to reporters. See https://www.nytimes.com/2017/06/29/business/media/new-york-times-staff-members-protest-cuts.html https://www.nytimes.com/2017/06/29/business/media/new-york-t...
- jstewartmobile 9y agoJournalism is in the toilet, so journalists end up being one-man bands. Local Gannett paper had everyone using CRTs--in 2014!!!
- ax0ar 9y agoNot true. Presenting is part of writing. It's not separate.
- mozumder 9y agoA lot of tech people don't get this. You see this in web design, where they think HTML and CSS are separate domains. They think CSS is just pretty effects. Style can convey as much information as text.
- jstewartmobile 9y agoNot true that journalism is in the toilet? My wife was, and all of her friends still are, journalists. They constantly worry and complain about all of the tasks that were once the domain of trained professionals--photographers, editors, designers--being put on the journalist due to budgetary constraints.
- atonse 9y agoI've invested in Mobiledoc (mostly because it was for two ember apps) and it's mostly been fine, but is there any reason why I shouldn't be moving to ProseMirror? Seems to be a lot more powerful and have things like collaborative editing and track changes as well.
- JasonSage 9y agoJust my experience: I had a rough time with ProseMirror trying to use it before it was released. It didn't use modern JS modules for a long time, the source code was a mess as far as I could tell, and JavaScript variable errors would surface with every version. Maybe the author has overcome these issues, but I'd be inclined to go with a different editor. Also, ProseMirror's architecture is extremely close to DraftJS, which I've had much fewer issues with.
- marijn 9y agoYeah thanks for dropping some deeply outdated impressions in here. (Also not even fair—the code was never a mess and at the time it wasn't clear how ES modules would be used so using CommonJS was a completely reasonable choice.)
- JasonSage 9y agoImpressions matter, no matter how outdated they are. I also don't think the concept of fairness applies since I'm accounting the details of my personal experience and the conclusions I personally drew from it. Some of those are subjective (code quality) while others are purely objective (I got a lot of errors). What I infer from that may be entirely different from what you do, but that does not make your characterization of things more fair than mine.
- vertex-four 9y agoDoes a review of Windows 3.1 give you any idea at all of how applicable Windows 10 would be to solving a given problem? At best, you’ve stated a bunch of nonsense for people to filter out once they eventually realise you’re talking about a version which bears little similarity to the current state of things.
- Phrodo_00 9y agoThis isn't even a test editor. It's closer to a web publishing tool. The article also didn't talk about the normal text editor challenges like the way the text is stored in memory and modified.
- gbrown_ 9y agoAgreed. I carry the personal view that when someone says "text editor" they are implicitly implying plain text. As such the leading paragraph of the article makes me twinge a little when "text editor" is used rather than "word processor". The article then continues on to describe the editor which sounds very much like a WYSIWYG editor. Indeed ProseMirror[1] homepage says the following. ProseMirror tries to bridge the gap between Markdown text editing and classical WYSIWYG editors. It does this by implementing a WYSIWYG-style editing interface for documents more constrained and structured than plain HTML So whilst I've made a point of it reading oddly (and I would have thought this would also be the view of most others on HN) I am not trying to disparage the authors use of the term "text editor". But rather I am curious as to why such nomenclature happened to be used here as I seldom see it outside of descriptions of things such as Vim and Emacs. [1] http://prosemirror.net/ http://prosemirror.net/
- tedmiston 9y agoNot sure why you're getting downvoted. This appears to be an evolution of WYSIWYG tools to me, which I wouldn't categorize as a text editor either — more like a word processor. FWIW ProseMirror does not bill itself as a text editor: > An ideal content editor ... bridg[ing] the gap between Markdown text editing and classical WYSIWYG editors. > It does this by implementing a WYSIWYG-style editing interface for documents more constrained and structured than plain HTML... from https://prosemirror.net https://prosemirror.net
- zmix 9y agoXML
- AJRF 9y agoI'd love to work for this engineering team. Their tech blog and github is full of interesting projects.
- zem 9y agothe BBC sounds like it has a really good tech team too. wonder what it is about newspapers that leads to this sort of tech culture.
- jjeaff 9y agoI'd say it's desperation. The traditional business model of newspapers has been upended, and the ad driven model isn't really working out that well either. So instead of doubling down and riding it to the ground, some outlets are actually innovating and making cool stuff and throwing it against the wall to see what happens.
- SwellJoe 9y agoWhile I agree, I wouldn't think BBC would be dealing with those same pressures, being primarily (I think?) state-funded.
- richardcrossley 9y agoThe BBC is state funded, through the TV Licence, for use within the United Kingdom, elsewhere it is funded on a commercial basis. The web site differs between domestic (no adverts) and international (adverts). BBC Studios is the commercial arm of the BBC for licencing and distributing media outside of the United Kingdom. https://www.bbcstudios.com/about/about-us/ https://www.bbcstudios.com/about/about-us/
- slobotron 9y agoAn excellent profiler for perl, NYTProf [0], came out of New York Times. It's still actively developed and still kicks ass - outputs easy to grok flame graphs that you can click around on to zero into any statement. More than just finding bottlenecks, I find it useful to understand workings of new codebases... [0] https://metacpan.org/pod/Devel::NYTProf https://metacpan.org/pod/Devel::NYTProf
- smacktoward 9y ago> this new story editor needs to combine the advanced features of Google Docs with the intuitive design focus of Medium This part made me sad. Imagine how much more powerful our tools could be if we didn't insist on jamming them all into web browsers!
- amelius 9y agoBut at least they run everywhere.
- noir_lord 9y agoWhere everywhere is some subset of browsers in the wild and often not that consistently anywhere. I mean the web is awesome but it’s far from perfect from a programming point of view but that of course doesn’t matter for other reasons.
- crcl 9y ago> Where everywhere is some subset of browsers in the wild and often not that consistently anywhere. Yes, but it's easier to download a version of the browser that works. The alternative is for the developer to build native apps for each OS they want to support. In my experience, it's easier to build cross-browser than it is to build cross-platform.
- jopsen 9y agoIf you have professional writers why not just teach them markdown and git? Instead of spending engineering $$$ building, maintaining and operating a complicated editor?
- ktpsns 9y agoBecause professional writers are not professional coders. Markdown covers the basic text markups but does not define how to format all these embedded media a modern newspaper has. Just to give an example: Inspect the syntax of wikipedia, which is the MediaWiki wikitext (https://www.mediawiki.org/wiki/Wikitext https://www.mediawiki.org/wiki/Wikitext) language. It is simple and therefore Wikipedia pages are rather simple. I expect the NYtimes to have higher aims in markup.
- combatentropy 9y ago>> why not just teach them markdown and git? I myself am surprised they don't just use markdown. Git is another story. > writers are not professional coders. But I have heard of writers clinging to old versions of WordPerfect, which were not WYSIWYG. They preferred the keyboard shortcuts to clicking around with the mouse. Furthermore if you're just writing the text of the article, you don't even need markdown, just a textarea. They used to do these on typewriters after all. Newspaper articles traditionally lacked anything but plain text. No bold, italics, or subheadings. > Markdown covers the basic text markups > but does not define how to format all these embedded media > a modern newspaper has Traditionally the jobs were separate --- and I assume they still are. There are the reporters, who research and write the text of the article. Then there is the layout team who lay it out, decorate it, etc. I can see how markdown would be no good for them. They would want a graphical user interface.
- acdha 9y ago> Traditionally the jobs were separate --- and I assume they still are. Is this really a good assumption? Journalists are making complex documents with central interactive features and lots of data visualization. It seems a lot like classic waterfall thinking to assume that reporters could perfectly envision that and toss it over the wall for a separate team.
- kaennar 9y agoOne of the most truly fascinating things I learned from my father was how early 1980s word processors were designed for the magazine he worked for. The different ways to handle fonts, screen placement, and how to include data from a local database. It was truly an awakening to the hidden wonders of the world.
- briandear 9y ago"We’re planning to begin work soon on a collaborative editing feature that would allow more than one user to edit an article at the same time" You mean like Google Docs or Quip?
- rayalez 9y agoI'm choosing an editor for my project right now. Can someone compare ProseMirror to Slate and Quill? What is the difference, which one is better? What do I need to know to choose one?
- FMashiro 9y agoIf you just need a WYSIWYG editor on a web page, and it's a single one, I've used tinyMCE. On projects where I've had to load up multiple instances, I've had to move away form it as it kept reloading stylesheets and js files for each editor on the page. Now I usually use quilljs, which works really well for some basic markup.
- onurozkan 9y agoProsemirror and Slate both great projects but lack of community interest. They are basically framework for creating WYSIWYG, until they release full featured editor built on it, i dont think it will become popular. Personally tried both, working with Slate feels much smoother, plugin system easier than prosemirror in my opinion. And, there is ckeditor5-beta. You might wanna check out.
- chmike 9y agoAnd couldn't find any info on the backend requirement. Do they require nodeJS or can any backend framework be used ?
- pfooti 9y agoI've used a lot of quill in the past couple of years. It has some tricky bits, but seems similar in a lot of ways to ProseMirror - separate internal representation of data that gets rendered into HTML, delta-based state changes, etc. At this point, I use quill out of momentum - it was the only game in town when I started working with rich text, and now I know how it works. I also like it enough that I've never felt the need to go find something else; it's always done what I want. I think to fully experience all three of those editors, you should build the following: something that generates a rich text input using the editor, updates a data store (can be just in-memory if you want), and then re-presents the rich text in a read-only format. With Quill in particular, there's two ways to approach the second part - you could dump the data into another instance of the quill editor, but one flagged as read-only, and let quill handle rendering out all the elements. I don't do that, myself, as it's a bit heavy and relies on the DOM to be present to create the end result (meaning rendering rich text into emails on the server side can be tricky). Instead I use a library (that I wrote) that converts the quill representation into an HTML string that can be later injected into whatever place it needs to go. In general, I think that's where the one of the important pain points is for any major modern text editor exist - converting the internal data representation into a canonicalized HTML string. This gets further complicated by your framework - it's easy enough to inject an HTML string, and angular 1 lets you even compile that so you can have custom elements there, but angular 2 has no such facility (compiling; you can still inject). So rendering custom elements into angular 2 is feasible but quite tricky. You'll need to figure out how to handle custom elements too, like at-mentions and so on; those are really domain-specific. How I handle at-mentions in quill (with a pop-up autocomplete) depends on the structure of the overall system as well as the framework I'm in - angular 1, 2 and react all suggest different approaches for this. Anyway, this is all a long response that means "it depends". Each editor has its own strengths and weaknesses that are exposed by the specific data and presentation needs of your project and the other frameworks in play. But building an atmention or other custom element and converting from the internal data structure to HTML is a good test of functions for all of those.
- TeMPOraL 9y agoRE the flat paragraph structure - did they just reimplement Emacs in the browser? https://www.gnu.org/software/emacs/manual/html_node/elisp/Text-Properties.html https://www.gnu.org/software/emacs/manual/html_node/elisp/Te... (Also I think - from occasional messing with XML in .docx files - but can't be sure that Word flattens styling like that as well.)
- nwmcsween 9y agoAny sufficiently complicated editor contains an ad-hoc, informally-specified, bug-ridden, slow implementation of half of emacs? I prefer vim though..
- TeMPOraL 9y agoVim ain't sufficiently complicated (that can arguably be seen as a good thing). That's why it's been mostly reimplemented in Emacs.
- nwmcsween 9y agoVim has vimscript (ugh) with a lua and python gaining popularity so emacs sort of got it right IMO anyways.
- thomastjeffery 9y agoNeovim's main goal seems to be replacing vimscript with lua in a backwards-compatible way.
- zaarn 9y agoAny editor is simply a graphical frontend to ed, even vim.
- nojvek 9y agoIs Oak going to be opensourced? Looks very interesting.
- econnor 9y agoI'm speechless.
- netzone 9y agoSo, we had this product (or the larger one anyway) demoed for my work and our takeaway was that only really large organisations could possibly use it. It needs to many human hands between writing an article and getting it published. We want to move to something more automated, and this system would have been a serious setback.
- priyadarshy 9y agoI've been watching this product for a while: https://www.nuclino.com/ https://www.nuclino.com/ it seems they too have built something really impressive on top of ProseMirror (it's a lot like Quip/Paper but more interesting). ProseMirror really feels like the only editor on top of which you can build an entire user experience and app.
- wemdyjreichert 9y agoOak was the codename of Java IIRC... May confuse some old people like me
- dugmartin 9y agoI'm surprised she didn't mention Marijn in the "thank yous" at the end - without him giving away his time building ProseMirror they would have nothing to build upon.