4 ms·

In what ways is EDGAR elegant?

piker 9y ago
Well, for one, it's 34 years old and is simple enough for non-technical people to use but robust enough for institutional use. For another, it's scaled to accommodate 3 decades of exponential growth in public filings and regulation. Did I mention it's 34 years old?
- deleted 9y ago
  [deleted]
- Harkins 9y ago
  I'm not sure about exponential growth in public filings (and EDGAR does not track regulation, so it's not relevant). For example, the number of publicly traded companies is down about 70% compared to twenty years ago: https://fred.stlouisfed.org/series/DDOM01USA644NWDB https://fred.stlouisfed.org/series/DDOM01USA644NWDB Could you share a link showing exponential increase in the use of EDGAR?
  - metaobject 9y ago
    While there may be fewer companies, are there also fewer stock transactions, etc? This system tracks a lot of different types of transactions involving public companies.
  - piker 9y ago
    Sure. Although you should be aware that not only public companies are required to make filings with the SEC. Index - Q1 1994 - 3mb - https://www.sec.gov/Archives/edgar/full-index/1994/QTR1/form.idx https://www.sec.gov/Archives/edgar/full-index/1994/QTR1/form... Index - Q1 2017 - 45mb - https://www.sec.gov/Archives/edgar/full-index/2017/QTR1/form.idx https://www.sec.gov/Archives/edgar/full-index/2017/QTR1/form...
    - hbcondo714 9y ago
      File sizes are also increasing because filers are adding structured data[1] to their SEC filings. Also, most SEC Filings are now verbose HTML files instead of plain-text files. [1] https://www.sec.gov/structureddata/what-is-structured-data https://www.sec.gov/structureddata/what-is-structured-data
      - zwp 9y ago
        And not really exponential. Something interesting happened 2003-2004 though. We can perhaps observe lead up to dot-bomb (1996-1999 growth) and the 2008 financial crisis. http://hpics.li/2e78909 http://hpics.li/2e78909 1994,3153191 1995,4813587 1996,7539137 1997,13755958 1998,16107028 1999,15935643 2000,17548021 2001,16873202 2002,18904001 2003,27723307 2004,47116841 2005,47982675 2006,50672891 2007,51322342 2008,49635521 2009,45312542 2010,45384720 2011,46455461 2012,46733150 2013,45842099 2014,47065954 2015,48112988 2016,46636963 2017,47037415
        hbcondo714 9y ago
        Nice breakdown! The SEC introduced some new filing requirements[1] in 2003 and 2004 that may have contributed to this. [1] https://www.sec.gov/info/edgar/regoverview.htm https://www.sec.gov/info/edgar/regoverview.htm
      - piker 9y ago
        It's an index - a '\n'-delimited list of strings indicating the locations of filings. The size of those filings has nothing to do with the size of the index.
  - whatok 9y ago
    EDGAR does not only contain public company filings.
  - huac 9y ago
    Additional regulation increases the number of filings that must be made.
- berberous 9y ago
  EDGAR is admirable in many ways, but it's age is really showing, especially from a UI and UX perspective. I really wish the government would spend some effort in modernizing that aspect of it. BamSEC.com charges $35/month for essentially just a prettier UI, and it's so much more pleasant to use. Honestly, it's a pity the government never does M&A -- they should just buy BamSEC. The public filings themselves are also a nightmare of hard to follow formatting.
  - neuland 9y ago
    Thanks for mentioning BamSEC.com. It looks way easier to use, even if it is a paid service.
    - hbcondo714 9y ago
      Try out Last10K.com for SEC Filings. Admittedly it's not as "pretty" but it's free and includes some unique features including sentiment analysis. Disclosure: Last10K.com is a side project of mine
  - piker 9y ago
    Probably the product of (1) multiple changes in "best practices" data formats over 30 years (XML seems to be the format de jure), (2) optimizing for the filer and (3) diversity of filing data. Agreed it would be ideal if they were all similarly structured, but they have another set of stakeholders on whom they are imposing essentially a tax -- the filers themselves -- who have a legitimate gripe any time the SEC imposes a switching cost. It's easy to dismiss those gripes, but the vast majority of these filers aren't GOOG-sized. Many are very small, and those costs can be non-trivial to them. Edit: Worth mentioning here that EDGAR's ease of use makes sites like BamSEC simple to create. Probably EDGAR's time is best spent focusing on the institutional intermediary. That allows competition to iterate on the best UX/UI for browsing these filings without imposing the cost on the filers themselves as discussed above.
  - JumpCrisscross 9y ago
    > EDGAR is admirable in many ways, but it's age is really showing, especially from a UI and UX perspective I like the bare-to-the-bones simplicity. In contrast to "better designed" portals of some other countries, I can usually point even the most technically-illiterate executive to the filing portal and expect to get something useful out the other end. Disclaimer: I am not a lawyer. Please consult a securities lawyer before preparing and/or filing anything with the SEC.
deleted 9y ago
[deleted]
feelin_googley 9y ago
1. Easy to crawl i.e. simple HTML pages 2. Multiple open data formats, including TXT 3. Easy-to-follow organization (CIK), minimal indirection 4. No Javascript required, no gratuitous frameworks* 5. Bulk downloads of quarterly data (FTP up until 2016) *not to mention gratuitous iframes (BamSEC) Thanks to #5 anyone not satisified with the "UI/UX" can take the data and go build their own "UI". When data is tied up in layer upon layer of indirection, Javascript, iframes, enormous overstuffed URLs, binary formats (e.g. PDF, XLS), then others cannot as easily take the data and build their own interfaces. Not a good approach for public data nor good use of public expenditure, IMO. I am grateful to Carl Malamud for creating EDGAR.

Well, for one, it's 34 years old and is simple enough for non-technical people to use but robust enough for institutional use. For another, it's scaled to accommodate 3 decades of exponential growth in public filings and regulation. Did I mention it's 34 years old?

deleted 9y ago
[deleted]
Harkins 9y ago
I'm not sure about exponential growth in public filings (and EDGAR does not track regulation, so it's not relevant). For example, the number of publicly traded companies is down about 70% compared to twenty years ago: https://fred.stlouisfed.org/series/DDOM01USA644NWDB https://fred.stlouisfed.org/series/DDOM01USA644NWDB Could you share a link showing exponential increase in the use of EDGAR?
- metaobject 9y ago
  While there may be fewer companies, are there also fewer stock transactions, etc? This system tracks a lot of different types of transactions involving public companies.
- piker 9y ago
  Sure. Although you should be aware that not only public companies are required to make filings with the SEC. Index - Q1 1994 - 3mb - https://www.sec.gov/Archives/edgar/full-index/1994/QTR1/form.idx https://www.sec.gov/Archives/edgar/full-index/1994/QTR1/form... Index - Q1 2017 - 45mb - https://www.sec.gov/Archives/edgar/full-index/2017/QTR1/form.idx https://www.sec.gov/Archives/edgar/full-index/2017/QTR1/form...
  - hbcondo714 9y ago
    File sizes are also increasing because filers are adding structured data[1] to their SEC filings. Also, most SEC Filings are now verbose HTML files instead of plain-text files. [1] https://www.sec.gov/structureddata/what-is-structured-data https://www.sec.gov/structureddata/what-is-structured-data
    - zwp 9y ago
      And not really exponential. Something interesting happened 2003-2004 though. We can perhaps observe lead up to dot-bomb (1996-1999 growth) and the 2008 financial crisis. http://hpics.li/2e78909 http://hpics.li/2e78909 1994,3153191 1995,4813587 1996,7539137 1997,13755958 1998,16107028 1999,15935643 2000,17548021 2001,16873202 2002,18904001 2003,27723307 2004,47116841 2005,47982675 2006,50672891 2007,51322342 2008,49635521 2009,45312542 2010,45384720 2011,46455461 2012,46733150 2013,45842099 2014,47065954 2015,48112988 2016,46636963 2017,47037415
      - hbcondo714 9y ago
        Nice breakdown! The SEC introduced some new filing requirements[1] in 2003 and 2004 that may have contributed to this. [1] https://www.sec.gov/info/edgar/regoverview.htm https://www.sec.gov/info/edgar/regoverview.htm
    - piker 9y ago
      It's an index - a '\n'-delimited list of strings indicating the locations of filings. The size of those filings has nothing to do with the size of the index.
- whatok 9y ago
  EDGAR does not only contain public company filings.
- huac 9y ago
  Additional regulation increases the number of filings that must be made.
berberous 9y ago
EDGAR is admirable in many ways, but it's age is really showing, especially from a UI and UX perspective. I really wish the government would spend some effort in modernizing that aspect of it. BamSEC.com charges $35/month for essentially just a prettier UI, and it's so much more pleasant to use. Honestly, it's a pity the government never does M&A -- they should just buy BamSEC. The public filings themselves are also a nightmare of hard to follow formatting.
- neuland 9y ago
  Thanks for mentioning BamSEC.com. It looks way easier to use, even if it is a paid service.
  - hbcondo714 9y ago
    Try out Last10K.com for SEC Filings. Admittedly it's not as "pretty" but it's free and includes some unique features including sentiment analysis. Disclosure: Last10K.com is a side project of mine
- piker 9y ago
  Probably the product of (1) multiple changes in "best practices" data formats over 30 years (XML seems to be the format de jure), (2) optimizing for the filer and (3) diversity of filing data. Agreed it would be ideal if they were all similarly structured, but they have another set of stakeholders on whom they are imposing essentially a tax -- the filers themselves -- who have a legitimate gripe any time the SEC imposes a switching cost. It's easy to dismiss those gripes, but the vast majority of these filers aren't GOOG-sized. Many are very small, and those costs can be non-trivial to them. Edit: Worth mentioning here that EDGAR's ease of use makes sites like BamSEC simple to create. Probably EDGAR's time is best spent focusing on the institutional intermediary. That allows competition to iterate on the best UX/UI for browsing these filings without imposing the cost on the filers themselves as discussed above.
- JumpCrisscross 9y ago
  > EDGAR is admirable in many ways, but it's age is really showing, especially from a UI and UX perspective I like the bare-to-the-bones simplicity. In contrast to "better designed" portals of some other countries, I can usually point even the most technically-illiterate executive to the filing portal and expect to get something useful out the other end. Disclaimer: I am not a lawyer. Please consult a securities lawyer before preparing and/or filing anything with the SEC.

deleted 9y ago

[deleted]

feelin_googley 9y ago

1. Easy to crawl i.e. simple HTML pages 2. Multiple open data formats, including TXT 3. Easy-to-follow organization (CIK), minimal indirection 4. No Javascript required, no gratuitous frameworks* 5. Bulk downloads of quarterly data (FTP up until 2016) *not to mention gratuitous iframes (BamSEC) Thanks to #5 anyone not satisified with the "UI/UX" can take the data and go build their own "UI". When data is tied up in layer upon layer of indirection, Javascript, iframes, enormous overstuffed URLs, binary formats (e.g. PDF, XLS), then others cannot as easily take the data and build their own interfaces. Not a good approach for public data nor good use of public expenditure, IMO. I am grateful to Carl Malamud for creating EDGAR.