Page 5 of 7 FirstFirst ... 34567 LastLast
Results 41 to 50 of 63

Thread: Trading halted on NYSE floor

  1. #41
    u mad, snowflake?
    ReverendHellh0und's Avatar
    Join Date
    Sep 2007
    Location
    TITANPOINTE, NYC
    Last Seen
    Today @ 02:20 PM
    Gender
    Lean
    Libertarian
    Posts
    68,651

    Re: Trading halted on NYSE floor

    Quote Originally Posted by brothern View Post
    The NYSE's trading is halted all the damn time due to technical glitches. It's happened at least 4 times this year that I can remember.

    Link please.


    3.5 hours?


    Matthew 10:34
    Think not that I am come to send peace on earth: I came not to send peace, but a sword.

  2. #42
    Sage
    Gaius46's Avatar
    Join Date
    Dec 2012
    Location
    New York
    Last Seen
    Today @ 05:16 AM
    Gender
    Lean
    Libertarian
    Posts
    6,694

    Re: Trading halted on NYSE floor

    Quote Originally Posted by ReverendHellh0und View Post
    What do you think United's operational uptime percentage is? The NYSE? Why would the NYSE engage in an upgrade mid week? Why did that anonymous tweet seem to be in the know?

    yeah it was only the dog and pony trading floor, but its far too coincidental to believe the reports.


    If my system went down for 3 hours, people would be fired and providers would be sued. we operate above that %99.999 threshold. do you think the NYSE or united, or the WSJ would operate any less efficient?
    Emergency release to fix something else that it was decided couldn't wait until the weekend. I don't know that that's what happened but it's an often enough occurrence.
    Or how about an upgrade over the weekend that didn't actually exercise the problem code path until yesterday?

    How do you define your 99.999% uptime? I ask because it's impossible to make it meaningful in the real world for any moderately complex system.
    Last edited by Gaius46; 07-09-15 at 01:08 PM.
    Quo usque tandem abutere, Trump, patientia nostra?

  3. #43
    Sage
    Gaius46's Avatar
    Join Date
    Dec 2012
    Location
    New York
    Last Seen
    Today @ 05:16 AM
    Gender
    Lean
    Libertarian
    Posts
    6,694

    Re: Trading halted on NYSE floor

    Quote Originally Posted by ReverendHellh0und View Post
    wait this is a hoot....


    "NYSE and NYSE MKT began the process of canceling all open orders, working with customers to reconcile orders and trades, restarting all customer gateways and failing over to back-up trading units located in our Mahwah, NJ datacenter so trading could be resumed in a normal state"


    Mahwah is the backup now? (it's not, SHHHHH)

    and it took them 3 ****ing hours to fail over?

    Many of my customers have fault tolerance and redundancy, "failover" ranges from near instant, to about 10 mins.
    If it's a software problem all the hardware in the world won't help you.
    Quo usque tandem abutere, Trump, patientia nostra?

  4. #44
    Guru
    brothern's Avatar
    Join Date
    Jan 2013
    Location
    Chicago, IL
    Last Seen
    12-03-16 @ 03:42 PM
    Gender
    Lean
    Centrist
    Posts
    2,900
    Blog Entries
    8

    Re: Trading halted on NYSE floor

    Quote Originally Posted by ReverendHellh0und View Post
    Link please. 3.5 hours?
    Yes. Here's the two most recent:

    Trading Issue Affects More than 150 NYSE Arca ETFs - MoneyBeat - WSJ
    AEX options on Euronext halt over "technical issue" | SmartBrief
    Help fight Zika, TB, HIV/AIDs and water pollution by donating your CPU's excess processing time to computing-based scientific research.

  5. #45
    u mad, snowflake?
    ReverendHellh0und's Avatar
    Join Date
    Sep 2007
    Location
    TITANPOINTE, NYC
    Last Seen
    Today @ 02:20 PM
    Gender
    Lean
    Libertarian
    Posts
    68,651

    Re: Trading halted on NYSE floor

    Quote Originally Posted by Gaius46 View Post
    Emergency release to fix something else that it was decided couldn't wait until the weekend. I don't know that that's what happened but it's an often enough occurrence.
    They didn't call it an "emergency", they seemed to suggest it's how they operate. I don't believe it.


    How do you define your 99.999% uptime? I ask because it's impossible to make it meaningful in the real world for any moderately complex system.
    In laymans terms, that the **** works,

    They seem to think it's important:
    NYSE, New York Stock Exchange > About Us > News & Events > News Releases > Press Release 10-23-2013
    NYSE, New York Stock Exchange > About Us > News & Events > News Releases > Press Release 02-02-2012
    NYSE, New York Stock Exchange > About Us > News & Events > News Releases > Press Release 11-05-2012


    So do you really think this HA, redundancy, and fault tolerance they seem to be VERY big on, goes out the window for the floor?

    Do you think united is as haphazard about thier noc as well? Never mind it was the same day....

    An Inside Look: United Airlines' Mission Control Center - Forbes


    Matthew 10:34
    Think not that I am come to send peace on earth: I came not to send peace, but a sword.

  6. #46
    Educator
    Join Date
    Sep 2014
    Last Seen
    07-18-15 @ 12:26 PM
    Gender
    Lean
    Liberal
    Posts
    731

    Re: Trading halted on NYSE floor

    Quote Originally Posted by ReverendHellh0und View Post
    If my system went down for 3 hours, people would be fired and providers would be sued. we operate above that %99.999 threshold.
    That may well be true, but without knowing what sort of system you manage, I can't say whether it has the kind of complexity, traffic and the very narrow fault tolerances something like the NYSE does. If one trade in a million fails to complete for a technical reason, that's a huge, show-stopping, problem for the NYSE. Even if certain transactions just take 100 milliseconds longer to complete than they should, that is like alarms start going off an people panic. They can't rely on the usual techniques of letting things fail over after things time out or using not-quite-synchronous data and whatnot.

    Also, the system you manage is probably more similar to a lot of other systems, so you have to deal with bugs in the slice of the environment that is custom built code for that particular purpose, but most of the code you're using is like Apache, Linux, Windows, Oracle, etc.- things that have been tested and fixed up for many years and which serve millions of users. In the NYSE, the odds are nearly all their code is custom. Even most their hardware and network protocol is custom.

    So, I have no idea. It could have been Anonymous for all I know. But I wouldn't go around comparing uptime you see in a typical IT situation to uptime at something like the NYSE. A lot of what you would count as "up" would count as "down" for the NYSE and they're working in a much more complex, bespoke, environment.

  7. #47
    u mad, snowflake?
    ReverendHellh0und's Avatar
    Join Date
    Sep 2007
    Location
    TITANPOINTE, NYC
    Last Seen
    Today @ 02:20 PM
    Gender
    Lean
    Libertarian
    Posts
    68,651

    Re: Trading halted on NYSE floor


    Neither of these small glitches seemed to have involved mid week "upgrades" and forgetting to make sure the clients were compatible. while yes of course they may at times have issues, it does happen, the "we did an upgrade" doesn't make much sense.


    Matthew 10:34
    Think not that I am come to send peace on earth: I came not to send peace, but a sword.

  8. #48
    Sage
    Gaius46's Avatar
    Join Date
    Dec 2012
    Location
    New York
    Last Seen
    Today @ 05:16 AM
    Gender
    Lean
    Libertarian
    Posts
    6,694

    Re: Trading halted on NYSE floor

    Quote Originally Posted by ReverendHellh0und View Post
    They didn't call it an "emergency", they seemed to suggest it's how they operate. I don't believe it.




    In laymans terms, that the **** works,

    They seem to think it's important:
    NYSE, New York Stock Exchange > About Us > News & Events > News Releases > Press Release 10-23-2013
    NYSE, New York Stock Exchange > About Us > News & Events > News Releases > Press Release 02-02-2012
    NYSE, New York Stock Exchange > About Us > News & Events > News Releases > Press Release 11-05-2012


    So do you really think this HA, redundancy, and fault tolerance they seem to be VERY big on, goes out the window for the floor?

    Do you think united is as haphazard about thier noc as well? Never mind it was the same day....

    An Inside Look: United Airlines' Mission Control Center - Forbes
    You have a system that 20,0000 people connect to and has 15 functions. If one person cannot execute one function but everything else works is the system up or down? How does that failure get factored in uptime calculations?

    I'll refrain from commenting on anything specific to NYSE since I work for them - though not in the area affected yesterday. I will say we don't as general rule make routine production changes during the week. That doesn't mean we don't ever make them should circumstances warrant. It also doesn't mean anything that the release didn't use the words "emergency"
    Last edited by Gaius46; 07-09-15 at 01:25 PM.
    Quo usque tandem abutere, Trump, patientia nostra?

  9. #49
    u mad, snowflake?
    ReverendHellh0und's Avatar
    Join Date
    Sep 2007
    Location
    TITANPOINTE, NYC
    Last Seen
    Today @ 02:20 PM
    Gender
    Lean
    Libertarian
    Posts
    68,651

    Re: Trading halted on NYSE floor

    Quote Originally Posted by tuhaybey View Post
    That may well be true, but without knowing what sort of system you manage, I can't say whether it has the kind of complexity, traffic and the very narrow fault tolerances something like the NYSE does. If one trade in a million fails to complete for a technical reason, that's a huge, show-stopping, problem for the NYSE. Even if certain transactions just take 100 milliseconds longer to complete than they should, that is like alarms start going off an people panic. They can't rely on the usual techniques of letting things fail over after things time out or using not-quite-synchronous data and whatnot.
    We have clients who have similar requirements and have HA in place. we also don't do service upgrades in the middle of thier busy time.


    Also, the system you manage is probably more similar to a lot of other systems, so you have to deal with bugs in the slice of the environment that is custom built code for that particular purpose, but most of the code you're using is like Apache, Linux, Windows, Oracle, etc.- things that have been tested and fixed up for many years and which serve millions of users. In the NYSE, the odds are nearly all their code is custom. Even most their hardware and network protocol is custom.
    WE have both, and we manage several system, not jut one from our noc.


    So, I have no idea. It could have been Anonymous for all I know. But I wouldn't go around comparing uptime you see in a typical IT situation to uptime at something like the NYSE. A lot of what you would count as "up" would count as "down" for the NYSE and they're working in a much more complex, bespoke, environment.

    are you suggesting the stock exchanges networks are more apt to go down than say, United? oh wait. O


    Matthew 10:34
    Think not that I am come to send peace on earth: I came not to send peace, but a sword.

  10. #50
    Educator
    Join Date
    Sep 2014
    Last Seen
    07-18-15 @ 12:26 PM
    Gender
    Lean
    Liberal
    Posts
    731

    Re: Trading halted on NYSE floor

    Quote Originally Posted by ReverendHellh0und View Post
    We have clients who have similar requirements and have HA in place. we also don't do service upgrades in the middle of thier busy time.

    WE have both, and we manage several system, not jut one from our noc.

    are you suggesting the stock exchanges networks are more apt to go down than say, United? oh wait. O
    I mean, I don't know what kinds of systems you manage, so I can't really say, but here, take an example. Say that in one of the applications you manage, somebody submits a helpdesk ticket saying that their user profile page isn't loading. How long might it be before anybody checks into the ticket at all? If the page in fact is not loading, how long before somebody diagnoses the problem? How long before you get a fix on the system typically?

    Now, if instead of a user's page not loading, that was a situation where somebody's trade wasn't being recorded, that could leave the NYSE user in a situation where they lose $600,000 when the stock goes up 1% before they realize that the trade wasn't actually recorded. A similar severity of bug as might cause the user above's profile page not to load might cause a trade not to be recorded, but where in the profile page scenario, you consider the time that bug is in place to be uptime, and the bug may be in place for weeks, months or even years, the NYSE would immediately shut down and would not come back up until they found and fixed the bug and corrected any data that had been corrupted by it.

    Generally speaking, places that shoot for 99.999% uptime are places where users can deal with a few bugs and where the system behaving slowly for brief periods still counts as it being up. There are not a lot of environments that I'm aware of with 99.999% uptime where "up" means zero bugs and nearly instantaneous response times. And I can't think of any that hit that kind of zero bugs, instantaneous response, uptimes with a massive worldwide distribution of networks, servers and software doing extremely complex things running all custom software.

    For example, Google's search engine is massive and equally complex and it hits 99.999% uptime. But, it does it by being very fault tolerant. If you do the same search 100 times, you won't actually get quite the same results each time because different databases will be getting fed new data at different times and this thing won't be quite up to date on that one, but will on this other, etc. Many of the stats they give you, for example about search queries, the data you get back is in the form of "approximately 500 - 540" and whatnot because they're often using more scalable, faster, techniques when they don't need to be 100% confident in 100% accuracy. Other large complex systems often hit 99.999% by being slow. For example, most ecommerce platforms have no problem making a user wait even 5 or 10 seconds while they process things. The NYSE can't do either of those things, and I can't think of another equally complex system that can't do either of those things which hits 99.999% with no bugs.
    Last edited by tuhaybey; 07-09-15 at 01:41 PM.

Page 5 of 7 FirstFirst ... 34567 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •