5 ms·

We're using Jira, Confluence, and BitBucket on-prem at our DC now, on some pretty sizeable VM's with good network connectivity back to head office. Despite tha

by Jedd 6y ago

We're using Jira, Confluence, and BitBucket on-prem at our DC now, on some pretty sizeable VM's with good network connectivity back to head office. Despite that, it feels very slow. Not the occasional 10+ second page load, where you've accidentally clicked something that happens to generate a cascade of cache misses. I can forgive that kind of infrequent performance hiccup. But rather, regular usage, especially of Confluence and Jira, just feels sluggish when doing bread and butter actions - adding a comment to a ticket or reloading a kanban view, loading a new page or opening the search dialog on Confluence. Yet, everything I've heard about the responsiveness of the 'cloud' offering is that it drives people to move to the on-prem server version.

originalvichy 6y ago
I don’t share the same experiences with slowness (maybe we do something right on the instances we manage). I do however experience slowness in cloud. I assume they increase cloud resources in some kinds of steps that make them appear extremely sluggish.
- confluence_perf 6y ago
  Sorry to hear it's been a frustrating experience. I'm a PM for Confluence Cloud and we're always trying to make it better. Would you be willing to share more specifics, such as: - Pages with content X are the slowest - Trying to do A/B/C is annoyingly slow - etc ?
  - vasco 6y ago
    Surely you have metrics?
    - Bombthecat 6y ago
      If your avarage is always 10 seconds, 7 is fast!
    - confluence_perf 6y ago
      Hi vasco, We have a lot of metrics, and we recognize that Confluence is not as fast or as good an experience as it should be. However nothing beats user feedback - hearing specific feedback from users may provide us with specific areas that we can focus our work, on top of the work we're already doing. If you have any feedback like that, we're hopeful you're willing to share it with us.
      - vasco 6y ago
        I agree with you on user feedback being valuable, but not to figure out what's slow. You should be getting that information accurately through metrics and know what to prioritise without having to speak to anyone. Reserve questions you can't get answers for otherwise for user interviews.
  - ratherbefuddled 6y ago
    You should be able to experience the slowness for yourself by spending a few minutes using your product. Then try something like notion or pretty much any other web app and notice how your blood pressure drops and your stress levels decrease. It is not a particular area it is the whole thing. Every interaction makes me want to gouge my own eyes out. It's a glorified text area with a menu, why should every click cause dozens of requests that take more than 10 seconds to complete? I can only conclude Atlassian engineering is utterly disfunctional for this to have gone on so long unimproved.
    - confluence_perf 6y ago
      Hi ratherbefuddled, We definitely recognize we have a lot of work to do on performance in general, and we're definitely trying. "fixing everything" is definitely part of our plans, but we'd also like to "fix whatever users find the most frustrating". If any specific items stand out as the most frustrating please do let us know.
      - ratherbefuddled 6y ago
        I think you are taking entirely the wrong approach here. Pressing refresh on the front page causes TWO HUNDRED AND SIXTEEN requests for about 7MB of resources. The only meaningful content on this page is a list of 20 or so links. You don't need customer feedback here, you need some basic engineering principles and a tiny bit of pride.
        confluence_perf 6y ago
        Hi ratherbefuddled, Thank you for the insight. We are definitely working on generalized efforts to make 'most/all/everything' faster (including speeding up the app overhead), but those types of projects unsurprisingly take a little longer to finish. On the other hand, specific user feedback might be 'the most annoying thing to me is that pages with Links load slowly, I need the links to load earlier because of X', or 'I need inline comments to load faster than page comments, because of reason Y' -> this type of feedback could give us something we can fix quickly and alleviate user frustration sooner. Hopefully the intention makes sense. Of course, in general, we're also always looking for any feedback that can help us improve Confluence!
  - sheeeep86 6y ago
    Where is the data hosted? Somehow it feels like the devs are located close to the data and don't notice the lag. And for me in central Europe, some requests must travel across the world and back.
    - confluence_perf 6y ago
      Hi sheeeep86, My original post did not mean to give the impression we think Confluence is fast - we do recognize that we have a lot of room for improvement. The reason we're asking for feedback on specific areas that are annoying is that while we can work on making 'everything' faster, knowing which specific items may be of the most concern may allow us to focus fix those items first. If you do have any feedback around such specific items we would be happy if you're willing to share it.
pc86 6y ago
While we had very similar experiences - only used JIRA but the performance was borderline-unusable - I've heard reputable sources say that configuration plays a tremendous role in that. I'm inclined to believe we just hadn't configured it "correctly," but the fact that you can destroy performance by casual misconfiguration is itself a pretty big issue. I've heard the same about cloud performance, which seems to be an argument against it being simply configuration, but who really knows?
- Nullabillity 6y ago
  When I used JIRA Cloud about a year ago I measured 30s+ load times on every single page. For a nearly-empty instance with no customizations, just their scrum template.
  - ratherbefuddled 6y ago
    Yes, it's like that out of the box. The next gen stuff offers no improvement whatsoever.
tedivm 6y ago
The performance on the cloud version is awful. When I previously self hosted it was a miserable management experience, but it definitely felt snappier than the current cloud version does.
garenp 6y ago
Atlassian does not provide particularly good guidance on how to get their tools to perform well. The first big issue for any moderately busy system is that you need to dedicate huge gobs of heap memory (think 16G+) to the tool. After that, they suffer from poor schema design - there are tables that are literally missing indexes on commonly queried columns, and some columns that have delimited values embedded in them. It's hard for me to imagine that atlassian would allow individual instances of jira/confluence/etc in their cloud versions to get nearly as much resources as you need to give them on-prem. The incentive would always be for them to try to minimize the amount of memory every instance gets, which does NOT maximize your individual performance. Shared resources will also make performance more variable and "noisy", making it more difficult to narrow down specific causes. It's hard for me to see how "Java" and "cloud" go together.
- yjftsjthsd-h 6y ago
  > It's hard for me to imagine that atlassian would allow individual instances of jira/confluence/etc in their cloud versions to get nearly as much resources as you need to give them on-prem. I always wondered that, too! Like, you'd think that if you have devs building a product and a team running a hosted instance of the product in the same company, that they'd have a very good feedback loop and make it easy to run the thing and to do it well. Although, I wonder if the answer is that it's not the same product - like with bitbucket, where the thing they run in the cloud isn't the same thing as they sell for on-prem. Or even a lighter version (BB, AFAIK, is literally 2 separate codebases), where they cut features in order to make the cloud version work. > It's hard for me to see how "Java" and "cloud" go together. Nah, it's not Java, it's the specific product. I've worked at a SaaS shop that did all Java, and while the pathological cases are comically bad, it's actually fine most of the time IME.
  - tiew9Vii 6y ago
    They don’t. They forked the code base around 5 years ago now. Cloud/on prem. Cloud is or was multi tenanted, you didn’t get your own isolated server, you got routed to some compute that would then pull data at request time based on your tenant id. With a hard fork of two seperate product streams it was inevitable at some point maintaining feature parity between products would get to much. Java and cloud go together quite nicely. Backend services, a JVM that has now be tuned for many many years by lots of clever people. If you have a long lived service it works really well. It works better than your node js app running a single instance on a multi core server with the event loop getting blocked.
yjftsjthsd-h 6y ago
Yep:) At a previous job, we also did Jira/Confluence/BitBucket self-hosted, running on what should have been nice hardware, and spent months chasing performance issues, including a charming issue that was so bad that it broke jira for the entire company every... Monday, I think? When you have a well-used standard operating procedure for restarting Jira just to unfreeze it, you have a problem. We were pretty sure it was somebody running a weekly report with some monstrous query, but 1. not even Atlassian support could seem to tell us how to find such an event, and 2. I don't really care how bad the query is - if you can run a query and take down the system for the whole company, then the software is bad. And honestly, even "normal" performance was poor - I seem to recall that it at least acted like the web UI was the slow part; running a local CLI client against the API was faster.
- laurent92 6y ago
  Well they certainly noticed companies running their own Jira hurt the image of Atlassian, and stopped offering the service altogether. It would be a knee-jerk reaction but possible.
bootlooped 6y ago
Jira is probably the slowest piece of software I use in my job. I can't believe there's an interstitial loading screen with a status bar for cloning an issue.
smcl 6y ago
I've no recent experience of the on-prem version, but nowadays I have to use the cloud version and it's painfully slow. One common problem I encounter is that after a page loads I might want to search for something in the search box. I'll click it and start typing ... then at some point the rest of the page has loaded including the stupid key bindings which do various random actions on different keystrokes. I don't know why they have this, it's infuriating. Oh and they keep updating the UI which involves shuffling things around and every new version is slower than the last. If you're running on-prem you at least get to decide when to update and notify everyone accordingly. We'd just find out one day that everything has been moved around and learn the new locations of different features. Sorry I'm really just generally ranting about Jira now but honestly I find it so infuriating to use. Confluence as well, how did they manage to make wikis suck so bad?
- Jedd 6y ago
  Agreed. Hijacking ctrl-f in my browser (chrome) is user-hostile, given searching on-page is a reasonable expectation for any web app. I use Confluence regularly over a high latency (600ms) link - and this has the surprise / delayed page load you describe, with the newer search features refusing to action your search request until it's loaded & rendered a large overlay on the right-hand side of the screen. These two attributes make are hugely frustrating, given looking for information is the key function of a wiki.
ashtonkem 6y ago
My experience with Jira makes me think that extensions are a big portion of the problem here. Confluence just sucks though.
- RikNieu 6y ago
  What about Confluence sucks specifically? I share your view but how could it be better?
  - ashtonkem 6y ago
    1. Performance. It is incredibly slow without customization. 2. Editing is a clunky process. Not only is the UX subpar, it never loads all the possible widgets you can insert, making dynamic content a pain. 3. Awful search. It always manages to find internal process documents from 2014 and not the service pages I actually want. 4. Tough to reorganize content. Moving a page around is just hard enough that people tend not to do it, resulting in really poorly organized content. Honestly, I’d be happier using notion or just GitHub wikis. But the main benefit of putting our docs at wiki.$COMPANY.com is that I can guarantee discoverability for other teams, so we’re kinda stuck with it.

by Jedd 6y ago

originalvichy 6y ago

I don’t share the same experiences with slowness (maybe we do something right on the instances we manage). I do however experience slowness in cloud. I assume they increase cloud resources in some kinds of steps that make them appear extremely sluggish.

confluence_perf 6y ago
Sorry to hear it's been a frustrating experience. I'm a PM for Confluence Cloud and we're always trying to make it better. Would you be willing to share more specifics, such as: - Pages with content X are the slowest - Trying to do A/B/C is annoyingly slow - etc ?
- vasco 6y ago
  Surely you have metrics?
  - Bombthecat 6y ago
    If your avarage is always 10 seconds, 7 is fast!
  - confluence_perf 6y ago
    Hi vasco, We have a lot of metrics, and we recognize that Confluence is not as fast or as good an experience as it should be. However nothing beats user feedback - hearing specific feedback from users may provide us with specific areas that we can focus our work, on top of the work we're already doing. If you have any feedback like that, we're hopeful you're willing to share it with us.
    - vasco 6y ago
      I agree with you on user feedback being valuable, but not to figure out what's slow. You should be getting that information accurately through metrics and know what to prioritise without having to speak to anyone. Reserve questions you can't get answers for otherwise for user interviews.
- ratherbefuddled 6y ago
  You should be able to experience the slowness for yourself by spending a few minutes using your product. Then try something like notion or pretty much any other web app and notice how your blood pressure drops and your stress levels decrease. It is not a particular area it is the whole thing. Every interaction makes me want to gouge my own eyes out. It's a glorified text area with a menu, why should every click cause dozens of requests that take more than 10 seconds to complete? I can only conclude Atlassian engineering is utterly disfunctional for this to have gone on so long unimproved.
  - confluence_perf 6y ago
    Hi ratherbefuddled, We definitely recognize we have a lot of work to do on performance in general, and we're definitely trying. "fixing everything" is definitely part of our plans, but we'd also like to "fix whatever users find the most frustrating". If any specific items stand out as the most frustrating please do let us know.
    - ratherbefuddled 6y ago
      I think you are taking entirely the wrong approach here. Pressing refresh on the front page causes TWO HUNDRED AND SIXTEEN requests for about 7MB of resources. The only meaningful content on this page is a list of 20 or so links. You don't need customer feedback here, you need some basic engineering principles and a tiny bit of pride.
      - confluence_perf 6y ago
        Hi ratherbefuddled, Thank you for the insight. We are definitely working on generalized efforts to make 'most/all/everything' faster (including speeding up the app overhead), but those types of projects unsurprisingly take a little longer to finish. On the other hand, specific user feedback might be 'the most annoying thing to me is that pages with Links load slowly, I need the links to load earlier because of X', or 'I need inline comments to load faster than page comments, because of reason Y' -> this type of feedback could give us something we can fix quickly and alleviate user frustration sooner. Hopefully the intention makes sense. Of course, in general, we're also always looking for any feedback that can help us improve Confluence!
- sheeeep86 6y ago
  Where is the data hosted? Somehow it feels like the devs are located close to the data and don't notice the lag. And for me in central Europe, some requests must travel across the world and back.
  - confluence_perf 6y ago
    Hi sheeeep86, My original post did not mean to give the impression we think Confluence is fast - we do recognize that we have a lot of room for improvement. The reason we're asking for feedback on specific areas that are annoying is that while we can work on making 'everything' faster, knowing which specific items may be of the most concern may allow us to focus fix those items first. If you do have any feedback around such specific items we would be happy if you're willing to share it.

pc86 6y ago

While we had very similar experiences - only used JIRA but the performance was borderline-unusable - I've heard reputable sources say that configuration plays a tremendous role in that. I'm inclined to believe we just hadn't configured it "correctly," but the fact that you can destroy performance by casual misconfiguration is itself a pretty big issue. I've heard the same about cloud performance, which seems to be an argument against it being simply configuration, but who really knows?

Nullabillity 6y ago
When I used JIRA Cloud about a year ago I measured 30s+ load times on every single page. For a nearly-empty instance with no customizations, just their scrum template.
- ratherbefuddled 6y ago
  Yes, it's like that out of the box. The next gen stuff offers no improvement whatsoever.

tedivm 6y ago

The performance on the cloud version is awful. When I previously self hosted it was a miserable management experience, but it definitely felt snappier than the current cloud version does.

garenp 6y ago

Atlassian does not provide particularly good guidance on how to get their tools to perform well. The first big issue for any moderately busy system is that you need to dedicate huge gobs of heap memory (think 16G+) to the tool. After that, they suffer from poor schema design - there are tables that are literally missing indexes on commonly queried columns, and some columns that have delimited values embedded in them. It's hard for me to imagine that atlassian would allow individual instances of jira/confluence/etc in their cloud versions to get nearly as much resources as you need to give them on-prem. The incentive would always be for them to try to minimize the amount of memory every instance gets, which does NOT maximize your individual performance. Shared resources will also make performance more variable and "noisy", making it more difficult to narrow down specific causes. It's hard for me to see how "Java" and "cloud" go together.

yjftsjthsd-h 6y ago
> It's hard for me to imagine that atlassian would allow individual instances of jira/confluence/etc in their cloud versions to get nearly as much resources as you need to give them on-prem. I always wondered that, too! Like, you'd think that if you have devs building a product and a team running a hosted instance of the product in the same company, that they'd have a very good feedback loop and make it easy to run the thing and to do it well. Although, I wonder if the answer is that it's not the same product - like with bitbucket, where the thing they run in the cloud isn't the same thing as they sell for on-prem. Or even a lighter version (BB, AFAIK, is literally 2 separate codebases), where they cut features in order to make the cloud version work. > It's hard for me to see how "Java" and "cloud" go together. Nah, it's not Java, it's the specific product. I've worked at a SaaS shop that did all Java, and while the pathological cases are comically bad, it's actually fine most of the time IME.
- tiew9Vii 6y ago
  They don’t. They forked the code base around 5 years ago now. Cloud/on prem. Cloud is or was multi tenanted, you didn’t get your own isolated server, you got routed to some compute that would then pull data at request time based on your tenant id. With a hard fork of two seperate product streams it was inevitable at some point maintaining feature parity between products would get to much. Java and cloud go together quite nicely. Backend services, a JVM that has now be tuned for many many years by lots of clever people. If you have a long lived service it works really well. It works better than your node js app running a single instance on a multi core server with the event loop getting blocked.

yjftsjthsd-h 6y ago

Yep:) At a previous job, we also did Jira/Confluence/BitBucket self-hosted, running on what should have been nice hardware, and spent months chasing performance issues, including a charming issue that was so bad that it broke jira for the entire company every... Monday, I think? When you have a well-used standard operating procedure for restarting Jira just to unfreeze it, you have a problem. We were pretty sure it was somebody running a weekly report with some monstrous query, but 1. not even Atlassian support could seem to tell us how to find such an event, and 2. I don't really care how bad the query is - if you can run a query and take down the system for the whole company, then the software is bad. And honestly, even "normal" performance was poor - I seem to recall that it at least acted like the web UI was the slow part; running a local CLI client against the API was faster.

laurent92 6y ago
Well they certainly noticed companies running their own Jira hurt the image of Atlassian, and stopped offering the service altogether. It would be a knee-jerk reaction but possible.

bootlooped 6y ago

Jira is probably the slowest piece of software I use in my job. I can't believe there's an interstitial loading screen with a status bar for cloning an issue.

smcl 6y ago

I've no recent experience of the on-prem version, but nowadays I have to use the cloud version and it's painfully slow. One common problem I encounter is that after a page loads I might want to search for something in the search box. I'll click it and start typing ... then at some point the rest of the page has loaded including the stupid key bindings which do various random actions on different keystrokes. I don't know why they have this, it's infuriating. Oh and they keep updating the UI which involves shuffling things around and every new version is slower than the last. If you're running on-prem you at least get to decide when to update and notify everyone accordingly. We'd just find out one day that everything has been moved around and learn the new locations of different features. Sorry I'm really just generally ranting about Jira now but honestly I find it so infuriating to use. Confluence as well, how did they manage to make wikis suck so bad?

Jedd 6y ago
Agreed. Hijacking ctrl-f in my browser (chrome) is user-hostile, given searching on-page is a reasonable expectation for any web app. I use Confluence regularly over a high latency (600ms) link - and this has the surprise / delayed page load you describe, with the newer search features refusing to action your search request until it's loaded & rendered a large overlay on the right-hand side of the screen. These two attributes make are hugely frustrating, given looking for information is the key function of a wiki.

ashtonkem 6y ago

My experience with Jira makes me think that extensions are a big portion of the problem here. Confluence just sucks though.

RikNieu 6y ago
What about Confluence sucks specifically? I share your view but how could it be better?
- ashtonkem 6y ago
  1. Performance. It is incredibly slow without customization. 2. Editing is a clunky process. Not only is the UX subpar, it never loads all the possible widgets you can insert, making dynamic content a pain. 3. Awful search. It always manages to find internal process documents from 2014 and not the service pages I actually want. 4. Tough to reorganize content. Moving a page around is just hard enough that people tend not to do it, resulting in really poorly organized content. Honestly, I’d be happier using notion or just GitHub wikis. But the main benefit of putting our docs at wiki.$COMPANY.com is that I can guarantee discoverability for other teams, so we’re kinda stuck with it.