⚡ ENGAGE UPDATES ⚡
Learn about the impact of Engage technology, including
new capabilities and features, technical issues, and more.

Engage DNS Propagation Latency
Over this past weekend (03/25/22-03/27/22) we began a DNS migration project. All our DNS traffic is routed through our CDN Cloudflare. Typically new DNS propagation takes up to 24 hours, but is usually much shorter (4-6 hours). This was scheduled for over the weekend and on off-hours. However, we are getting some reports of service accessibility issues due to latency of DNS propagation. The symptom is that api.engage.co, directory.engage.co, and dashboard.engage.co may not be accessible to some users. We’re not sure why the DNS propagation is taking longer than expected to resolve for some users, but we do expect this to clear up on its own. Unfortunately this is one of the complications of the way that DNS server modification works: it can take time to update across the internet. Thank you for your patience.

Engage Elastic Search Latency
On Monday 3/21/2022 the Engage chat service started experiencing latency of Elastic Search. The symptoms were slow login and an impact to the displaying of chat widgets. We believe this was due to an unanticipated spike in traffic that put a heavy request load on key infrastructure, which subsequently affected the API service. Engineering spent a couple of hours on Monday night evaluating and resolving the issue. Unfortunately, it reared up again mid-morning on Tues (3/22). It is not down, but it is sluggishly slow to the point it may impact usability. We are again researching the issue and will provide an update soon. UPDATE: As of Tues at 11:53pm PT all Engage system were working properly. We discovered a previously unknown CPU/memory limit that we were bumping up against. As a result, we experienced 3 minutes of downtime on 3/20, 48 minutes of downtime on 3/21, and and 48 minutes of downtime on 3/22. However, the latency of our system caused many users issues in logging in, and the chat widget service was sporadic. The new memory limits were adapted once discovered last night and we fully expect all systems to be working well for the foreseeable future. Our deepest apologies for this previously undiscovered scaling issue and any impact it caused.

Engage API Service Non-Responsive
On Thursday January 13, 2022 at 8:20am PDT the Engage API service became unresponsive. We are running tests and will either A) restart the API service, or B) spin up a new service. We suspect this may have been caused by updates we pushed last night, but were not previously detected during staging testing. So, it’s either roll back or push forward. We’ll have one or other shortly. Thank you for your patience. UPDATE: 8:51 am PDT, we have detected the error. The chat history service (recall of all prior chat conversations) is having a hiccup due to other improvements we implemented. We have reset the service. But there may be some continued login latency while we make code updates over the next two days. We are investing heavily in service improvements. UPDATE: 10:02 pm PDT We pushed out all code changes and Engage is moving FAST! The clock speed on response times is faster than ever! We appreciate everyone’s patience with us as we worked double-time to get these new service enhancements rolled out. Hopefully your experience is seamless! 😉

Engage Dashboard Login Error
On Tuesday January 11, 2022 at 10:57am PDT some users of Engage have been unable to login, depending on their Region (AWS Zone). The result is that an attempt to login displays an “error” code. We are rebooting both of those zones and resetting login access. UPDATE: as of 11:29am PDT logging in should be fully functional across all zone.

Amazon AWS Down at 6:03am PDT
On Tuesday December 7, 2021 at 6:03am PDT Amazon AWS services were experiencing rolling outages impacting the US-East-1 region. We have not detected any downtime or impact for Engage services, but given the fact that we are on AWS and do use US-East-1 region (among others) we wanted to be proactive. It has impacted many services such as Slack, Netflix and others. While we do use US-East-1 region, we have load balancing and regional failover. So far, so good. We will provide updates if we detect any impacts.

UPDATE 2:51pm PDT 12/7/21. Unfortunately we did experience some latency and connectivity issues with AWS services in the US-East-1 region from approximately 2:04pm PDT to 2:46pm PDT, but then service levels returned to normal. Our understanding is that a number of services were knocked out up to five hours, so relatively speaking Engage has has less impact that others. That said, we apologize for any issues you experienced. It does appear that US-East-1 region is back online.

Engage API reset 11/8/21 at 8:02 CDT. If you are unable to login, please refresh your browser page https://dasboard.engage.co to login.

Engage API Unresponsive 10/26/21 - 10/27/21

On Tuesday October 26, 2021 at 6:09am PDT the Engage service became unresponsive for some customers. As a result, for users in certain regions it become sporadic whether or not they are able to login to the Engage chat dashboard. The symptom was that login attempts would result in the word “error” being displayed. Our engineering team is evaluating this regional access issue and unusually high API request volumes. If you are having any issues logging into your account, we do apologize and we are working to make the system responsive for you as soon as possible. Thank you.

UPDATE: as of 10/27/21 at 8:39am PDT this issue is still impacting some Engage users. We are continuing to evaluate the underlying issue causing authentication pathway errors.

UPDATE: as of 10/27/21 at 9:46am PDT we believe that Engage is experiencing a soft DoS attack. We have identified unprecedented traffic levels hitting our API. That traffic is originating from China, the Russian Federation, Mozambique and Malaysia. Historically when we see high numbers of API service requests we use our CDN (Content Delivery Network) to issue a JavaScript challenge. However, this traffic is sophisticated enough that it is satisfying the challenge and making the requests, which is putting extremely high demands on our infrastructure. We are currently implementing country blocks in the hopes that this stops the demands of this traffic. We can’t say it will with certainty, but it will take approximately 1 hour for us to start seeing any positive impact from adding the country blocks to our firewall. Again, we apologize for any inconvenience here. It’s very unfortunate that foreign bad actors are investing their life energy to disrupt the ordinary flow of commerce.

UPDATE as of 10/28/21 at 9:34am PDT the Engage API servers and chat directory service are experiencing rolling outages and failed web server requests. This is a particularly sophisticated DoS (Denial of Service attack). After originating in China and the Russian Federation, it continued extending to other countries. Over the last 24 hours over 80% of all this traffic originated inside of the U.S. (assuming that we’re not misidentifying the traffic / IP spoofing). While we have implemented JS challenges and country-level blocking, doing so for the U.S. would obviously stop utilization of the Engage chat service. We have been in contact with our CDN as well as filed a report with the Cybersecurity & Infrastructure Security Agency - Department of Homeland Security, explaining the economic impact that this DoS attack is having on the commercial communications of Engage clients and users. Our engineering team is continuing to do all we can to combat this DoS attack. For reference purposes, here is a reference of the normal activity level of our service vs. the DoS traffic inflicted upon Engage in the last 48 hours:

UPDATE as of 11/1/21 at 8:44am PDT the Engage API servers remain unstable. Our engineering team has been able to reduce by more than half the DoS traffic, so the loads on the Engage API have been such that some users have been able to sneak in and login. But, that has been more about lucky timing that service availability. We have two new proposed technology solutions that we plan to test and release later today. This “deduction by elimination” type of problem-solving is sub-optimal, but unfortunately this DoS traffic is extraordinarily sophisticated and extremely challenging to combat without the risk of also blocking the service for normal users/clients (we need to avoid throwing the baby out with the bath water). Thanks again for all the patience and words of encouragement we’ve received. We know how important the service is to your business and we’re doing everything in our power to get it back to our historically 99.99%+ availability.

UPDATE as of 9:46pm PDT 11/1/21 all Engage services are fully accessible. If you are having any issues logging in, please refresh your browser and re-login to https://dashboard.engage.co.

Engage Chat Servers Unresponsive Outage 9/7/21

On Tuesday September 7, 2021 at 7:44am PDT the Engage chat servers in the East Coast U.S. zone were detected as unresponsive. As a result, logging into Engage is blocked until a response from the chat servers occurs. We are researching load balancing to determine the cause.

Update: the East Coast zone was restarted at 8:10am PDT and logging in and chat communications should be working as expected.

Engage API Outage 8/16/21

On Monday August 16, 2021 at 6:44am PDT the Engage API service become unstable. We are diligently researching this issue. Symptoms include users/customers of the Engage Chat service are experiencing an issue where, upon logging into the chat dashboard, they were receiving an “error” message. This issue is impacting users/customers system-wide. We apologize for any inconvenience this is causing and we intend to have you back up and running ASAP.

Update: 10:45am PDT, most users are able to login now, although “previous chats” may not load immediately as our API service is still running sluggishly. We do expect for it to improve as the day moves on. The root problem is that there was an unusually high level of traffic from China and Russia in a very short period of time. We believe that the request volume caused the instability. We believe this may have been a lightweight DoS (denial of service) attack. By default we block traffic from China and the Russian Federation (in addition to several other nations), but a large volume of requests in a short period of time can still have a material impact on service delivery. We do apologize for any inconvenience this has caused, but it’s another great reminder of a business case for the U.S. to develop its own closed internet: to reduce technical and financial impact from the nefarious, bad actors, and nation-states determined to sow havoc. Unfortunately our openness is something they have chosen to take advantage of.

Engage Login Error 8/9/21

On Monday August 9, 2021 at 9:24am PDT users/customers of the Engage Chat service experienced an issue where, upon logging into the chat dashboard, they experienced slow load times for the dashboard. In addition, some chat widget were slow to load. An engineer is researching the network latency causing these slower response speeds

UPDATE: as of 11:52pm PDT the response times are back to expected levels with response times under 1,000 milliseconds.

Engage Login Error 6/3/21

On Thursday June 3, 2021 at 6:24am PDT users/customers of the Engage Chat service experienced an issue where, upon logging into the chat dashboard, they were immediately logged back out of the dashboard application. Our engineers is researching the login persistence issue at this time.

UPDATE: 6/3/21 7:53am PDT. We had an ES server on a node that needed to be restarted. All logins are working as expected.

Engage API Failure 3/9/21

On Tuesday March 3, 2021 at 2:24am PDT users/customers of the Engage Chat service experienced an issue where, upon logging into the chat dashboard, they were receiving an “error” message. This issue is impacting users/customers system-wide. Upon further investigation a critical failure of the Engage API was detected. The Engage engineering team is working on a rebuild and deploy of the API. We apologize for this inconvenience and we intend to have you back up and running ASAP.

UPDATE: 3/9/21 8:52am PDT the Engage API was relaunched. All logins, chat relay and widgets are responsive.

Engage CHAT LOGIN / AUTO-LOGOUT 10/16/20

On Friday October 16, 2020 at 9:04am PDT users/customers of the Engage Chat service experienced an issue where, upon logging into the chat dashboard, they were automatically logged out. This issue is impacting users/customers system-wide.

UPDATE: 10/16/20 10:41am PDT the Engage API was reset and capacity limits were increased. Logging in appears to be working as expected.

NOTE: The IP Geolocation Mapping Service in the Engage Dashboard is current not displaying a map. We’re working to update this to a more modern and fully featured map. Thank you!

Engage CHAT LOGIN LATENCY 8/20/20

On Thursday August 23, 2020 at 7:02am PST some users/customers of the Engage Chat service experienced problems due to load balancing issues with our API service which are due to an unusually large volume of requests to our service. The result has been delays in logging, gateway errors on the directory page, and sluggish (slows) load times for the chat dashboard. Our engineering team is aware of these issues and we’re working on updating the service. We apologize for any challenges you are experiencing in logging in.

NOTE: If you attempt to log in and get a spinning circle and/or a red “error” message, this is symptomatic of the problem with server requests being unanswered due to the load balance issue.

UPDATE: 8/20/20 3:03pm PST. Proxy routing was updated and all Engage systems and API are again available to all impacted customers. We do apologize to those customers who experienced downtime and/or login issues today. We’re still doing research on the spike. It does not appear to be a Denial of Service (DoS) attack, but we’re still not entirely certain about the source. We’re going to do follow-on research and we plan to implement new load balancing measures to help move affected customers onto new systems if/when an issue like this occurs in the future.

Engage CHAT SERVICE INTERMITTENT UNRESPONSIVE for maintenance 5/23/19

On Thursday May 23, 2019 at 9:02pm PST the Engage Chat service will be undergoing maintenance. The typical symptom is that a logged in user is logged out, and some who attempt to login are unable (the login appears to just spin while attempting to connect).

We are implementing updates. This is scheduled maintenance between 2019-05-24 04:00:00 UTC and 2019-05-24 06:00:00 UTC. During this time, the following Engage chat will mostly be unavailable.

Engage API NON-RESPONSIVE

On Monday January 28, 2019 at 12:31pm PST the Engage API was determined to be slow or non-responsive. The typical symptom is that, upon a login attempt, a message comes back saying “error” or “authorization failed”. This is a symptom of a non-responsive login attempt. It is very possible your credential are correct, but the response is erroneous due to the non-response from the API. We are monitoring the stability of the API service and may do a restart of the service. UPDATE: at 2:11pm PST we did a full restart syncing docs between multiple servers in the cluster ( and ran a full RECOVERING/INITIALIZING sequence). The relaunch was successful. All systems GO! All users should be able to login without issue.

Engage PRESENCE SERVICE 500 ERROR

On Tues January 15, 2019 at 6:54am PST the Engage presence service experienced a problem causing intermittent problems that are effecting all Engage customers. Users are still able to login to the dashboard, and the status still works on individual chat profile pages (user specific chat URL) but the presence service that instructs buttons and web widgets to show people online/offline has become unstable and in many cases is producing a 500 error. In these cases the button or widget will not show the one or more people who are currently available for conversations and, viewing the underlying reason for the error, it will produce Error: 500.” A 500 Internal Server Error is a very general HTTP status code that means something has gone wrong on the web site's server but the server. We are currently evaluating the presence service, including several servers, to detect the issue. While this particular service has never gone down before (EVER!), we have learned there is a first time for everything. We apologize for any inconvenience this has caused, including our client Syracuse University who beat #1 ranked Duke in basketball last night in a stunning overtime victory and who wants to make sure fans can get in touch. UPDATE: at of 10:02am PST on Weds Jan 16, 2019 a fix was pushed for this issue and all presence/status services were restarted and this appears to have addressed the issue of the presence outage.

Engage API outage 9/5/18

On Weds September 5 2018 12:41pm PST the Engage API was experiencing a massive outage.  This is impacted all parts of Engage including login, dashboard, and widgets.  This issue was corrected with an API restart at September 5 2018 at 1:05pm PT.

Engage DASHBOARD LOGIN ISSUE

On Tues August 7 2018 4:41pm PST the subdomain dashboard.engage.co (the login site) was not accessible. As of 7:09am CST on 8/8/2018 dashboard.engage.co was still inaccessible.  Two senior engineers and key architects of Engage are looking into the issue.  Under normal circumstances the dashboard page where you can login should load. However, for certain customers the page does not load and is resulting in a page load request timeout.  A reset of the web server occurred, but customers plagued with the page load timeout issue were still experiencing the issue.  Engage uses a CDN (content delivery network) the sits between our service and customers.  The CDN helps provider faster web page load times and more accessibility to Engage services from different markets.  It is presumed that there is a CDN configuration or service issue going on and we're researching this directly with our CND vendor. UPDATE: 8/8/18 2:02pm PST. A patch was released for the issue. Dashboard should be accessible and login should be immediately available.  Please note that if you're having trouble accessing dashboard.engage.co that it could be a delay in DNS propagation.  If so, please attempt accessing the site later and/or performing a shift>refresh of your browser window pointing to dashboard.engage.co.  Thank you and again, our most sincere apologies for those impacted by this most unfortunate issue.

Engage DASHBOARD LOGIN ISSUE

On Fri July 6 2018 4:19am PST the login site was not allowing logins to the Engage dashboard. This was reset at 7:04am PST and login is working as expected.

Engage API ISSUE

On Thurs June 14 2018 10:19am PST a critical issue was discovered in the Engage API.  The API has been unstable and certain affected customers have experienced problems logging in.  They symptom is an "error" message upon logging in.  We are researching the problem. For those of you having trouble logging in.

We discovered at 12:41am PST a widening and critical issue impacting subdomain pages as well, including profile and directory pages. Updates will be pushed as soon as we have resolution.

UPDATE: at 5:21pm PST a reset of all Engage infrastructure occurred.  We recommend refreshing your browser to avoid any potential cached issues.  The issues were caused by memory issues and we're looking into methods to combat/avoid this issue again.

Engage chatlog access error

On Tues April 17 2018 6:13am PST a critical issue was discovered in the Engage dashboard that prevents some users from viewing previous chats after they have logged out and logged back in.  This issue has been discovered to have effected some users going back as far as three weeks (to Mar 28 2018).  The symptom is that users have chats, log out of the dashboard, then re-login at a later date and recent chats do not display in the dashboard and are not discoverable in chat history search.  This is an extremely complex issue that also appears to be impacting chat forwarding (wherein you forward your chat logs to your email address of record).  This issue is currently in "review" status, but has been noted as an unusually difficult issue as it is affecting some users, but not others.  We are also interacting with a major vendor to better understand recourse measures.  We will detail more information as it comes available.  In the interim we ask that, if you believe you are being impacted by this problem, to consider manually copying your log for redundancy purposes.  Our most sincere apology for any incovenience this is causing any of you impacted.  We're very transparent about our flaws and this is one rare issue that we're finding extremely vexing. UPDATE: the problem was worked on and we believe a fix is working across all user accounts.  The week of 5/21/18 we also re-checked this and it appears that the fix is holding for all chat logging and conversation history access through the dashboard.

Engage api ACCESS

On Tues April 3 2018 7:14am PST the Engage API was experiencing connectivity problems.  The cause of this issue is related to a full cache.  The primary symptom is that those who are not currently logged in experience an "error" message when attempting to login to the Engage dashboard.  Our engineering group was been notified of the issue.  A restart of the cache was implemented and we believe logging in is working.  If you continue to receive an error, please attempt a browser refresh.  We apologize for any inconvenience. Thank you.

Engage api & DISK SPACE

On Tues January 2 2018 6:14am PST the Engage API was experiencing connectivity problems. In addition, a storm of new year user logins resulted in a disk space limit being reached on our XMPP server. We cleared logs and restarted all infrastructure and thus as of 9:54am PST it appears all systems are working as expected.  We have learned from some users that a browser or computer restart have helped with logging in, which could be related to browser caching.  We want to apologize for any inconvenience.  Wishing you a happy and successful 2018!

Engage api down

On Thurs December 22 2017 4:14am PST that Engage API was experiencing connectivity problems. One of the problems caused by this issue is an inability to login from the Engage dashboard; upon login attempt either a red "error" or "authentication failed" message shows on the dashboard login screen and does not allow you to login. Because of this problem users who are not able to login also can not connect to our presence (status) system and therefore will not show as "online".  Our CTO and a Senior Engineer were alerted to the problem, error logs were cleared, and the API was reset at 6:57am PST.  Engage should be fully functional for you as of the reset.

Potential Login issues

On Thurs December 14 2017 6:02am PST we were alerted that there have been regional challenges for some users logging in or staying connected to the Engage dashboard.  The symptom is that upon login either a red "error" or "authentication failed" message shows on the dashboard login screen and does not allow you to login, or the login takes a long time and when you do see the dashboard load the content is slow to display. Our CTO and a Senior Engineer reviewed the issue and successfully corrected the issue at 7:17am PST.

Login issues

There have been regional challenges for some users logging in or staying connected to the Engage dashboard.  The symptom is that upon login either a red "error" or "authentication failed" message shows on the dashboard login screen and does not allow you to login, or the login takes a long time and when you do see the dashboard load the content is NULL.  These issues appear to be connected to a major internet outages. The connection problems, which affected several major ISPs, including Comcast, Verizon, and AT&T, come just after the one-year anniversary of a DDoS attack on internet-infrastructure company, Dyn, that crippled the net for a day in 2016.  However, this week's outages are thought to have had a slightly more unremarkable cause: a misconfiguration at Level 3, an internet backbone company and enterprise ISP, that underpins other big networks.

Infrastructure Maintenance Notice

One or more of instances of Engage infrastructure in the Eastern U.S. Region are scheduled to be rebooted between Fri, 3 Nov 2017 07:15:00 GMT and Fri, 3 Nov 2017 10:00:00 GMT for required system maintenance in order to deploy important updates. Security and operational excellence are our top priorities, and therefore we occasionally need to do host maintenance. Each instance will experience a clean reboot and will be unavailable while the updates are applied. This generally takes no more than a few minutes to complete. Each instance will return to normal operation after the reboot, and all instance configuration and data will be retained.

Engage performs maintenance regularly to ensure that the service continues uninterrupted for our customers. In most cases, maintenance can be performed without service interruption. When maintenance cannot be performed without service interruption, we work hard to keep any impact as brief as possible. Thank you in advance for your patience and understanding.

POTENTIAL IMPACT notice from Engage

The Engage RESTful API service was experiencing downtime as of 7:53am PST, representing our first significant service interruption since November of 2016.  During this period you may be unable to login to the Engage dashboard, and/or logging in may be slow.  Another symptom is that the chat history that automatically populate may not display at all, or may be extremely slow to load.  Underlying hardware hosting our application in the us-east-1 region is experiencing degradation, and our server has been unreachable.  Our engineering team is researching the issue and we will post updates here as they become available.  Thank you and apologies for any inconvenience.

UPDATE 8:22am PST   Engage relies on Amazon AWS/EC2 services in the delivery of Engage.  Amazon has a major server/hardware instance failure.  We are looking to create and move to a new instance.

UPDATE 10:21am PST It appears that this server/hardware instance failure is catastrophic.  We are copying the instance, and are rebuilding the instance on completely new hardware.  This process may take up to 4 hours.  In the interim period you may have delays in logging in, you may not be able to login at all, certain data and analytics may not be available, directory services may not update promptly, and some or all site widgets (chat call-to-action elements) may not load on your primary or third-party websites.  Again, we apologize for this unforeseen issue experienced by our vendor Amazon AWS.

UPDATE 12:32am PST Multiple engineers at Engage are working at breakneck speed to create new instances of the failed servers.  There were several challenging dependencies and inter-dependencies in moving to a new instance and this work is in-progress.  We hope to relaunch the parts of Engage impacted by the failed servers in the next 1-2 hours.  It's all-hands-on-deck and thank you for your patience as we work through this extraordinary issue.

UPDATE 3:47pm PST Well, that's been fun.  We did get the work done.  Chat widgets are loading, company directories are working, the chat profile pages are loading, chat is working.  However, we're still noticing some minor impact issues, especially in the chat dashboard.

UPDATE: 7:05pm PST   Everything is up and running, but we're still seeing some "connection issues" in the chat dashboard relating to "recent conversations," "chat history," and "stats."  The recent conversations and any prior chat searchers are showing as empty (null) in some instances, and the stats reporting will sit and spin if you try to search a date range or keyword search.  We'll continue working through this issues.  Thank you again for your patience as we work through this major hardware failure at Amazon.

UPDATE: 6:03am PST. While it appears the vast majority of users are able to successfully log in to the chat dashboard, a few users continue to be impacted.  The following issues continue to be reported: 1) unable to login - it just 'sits and spins,' 2) the dashboard does not populate with my recent conversations, 3) the dashboard loads like normal but if I click on a recent conversation it sits and spins, 4) If I try to look up chat history or stats it does not populate with data and/or sits and spins.  We are still working to establish better service connectivity throughout the network.  We have rebooted all Engage services and taken a number of steps.  Engage also lives behind a global CDN (content deliver network) and we've confirmed that what's working is loading fast.  For those of you who were impacted yesterday we apologize. For those who continue to experience access issues, please know that we'll continue working to make sure we deliver the incredible service and uptime that we're known for.  Thank you!

UPDATE: 9:55am PST We have completely reconfigured our servers on the east coast node where Amazon AWS experienced the major hardware failure. To the best of our knowledge, all chat, dashboard, widgets, chat history and stats are functional. The only issue we are still seeing is the connection to browser preview.  Our heroic engineering team who have been working for 24 hours to rebuild instances are working on the browser preview rebuild, then they plan to get some well-deserved sleep. If you continue to have issues, please first try shift>refresh in your browser.  If things are still not working for you, please email us at support.  Thank you!

POTENTIAL IMPACT notice from Engage

Level 3 is one of the largest networks in the U.S. and they are having major outage issues today.  As a backbone to U.S. Internet infrastructure many ISPs rely on data transmission on Level 3.  The current issues they are experiencing could impact your use of Engage. The status of your users may not be transmitted, which could result in "error" issues in logging in as well as status not updating in widgets.  Engage infrastructure is up, but parts of the Internet are being impacted which would cause unusual behavior that could be experienced by some Engage clients.  Here is a map of the Level 3 outage area.  Engage was not impacted by the last major outage on 10/21/16 and we are not seeing any impacts today, but if your ISP uses Level 3 as a data carrier, that could explain the issues.  Also, we do not know for certain if this is another DoS attack similar to what occurred on 10/21/16, or if this is merely isolated to just Level 3.

A special notice from Engage

What is believeD to be a series of unprecedented DoS attacks has caused major impacts across the Internet today.  Major websites have been impacted at different points, and reports from the East Coast U.S. said at times half the Internet went dark.  It has been reported that the U.S. Government is looking at this as a criminal act.

We wanted to update you about Engage.  Our website engage.co has been impacted at several points and some visitors, especially those on the east coast, have not been able to access our site, or parts of our site.

However, none of our underlying services have been impacted.  The chat dashboard has been up continuously and all widget and chat API services remain at 100% uptime.  You can always view publicly the status of Engage infrastructure by visiting: http://stats.pingdom.com/7axuogji186z

Engage employs an extremely high level of security around our platform, and we support the uptime of our platform with distributed infrastructure across a global content delivery network.  While we fully expect our service to remain accessible to customers, we are continuing to monitor this situation closely.

Engage Error

• On Sunday, July 3rd we were alerted by a client to an issue where they manually logged out of Engage, shut their computer down, and received incoming chats over the evening. The symptoms were that when logging in the next day being surprised to see that incoming chats had occurred over the evening, that the timed-out response did not work, and that the customer contact info and customer information panel were not populating with customer data. We had not seen this issue previously and all initial tests on our end were unable to reproduce the issue (though we could observe the behavior in that user's account). The issue seemed to expand to several other users on 7.5.16 (for unknown reasons). As a result, we went all-hands-on-deck with all available engineering resources being applied towards addressing the issue. What we suspect right now is that this issue is being caused by our dashboard infrastructure correctly seeing the user has changed their status to offline, that information is being reported to our status server, but the status server is not updating the status to "release" the users status to "offline". A number of new test functions have been written to observe the behavior and a solution to the problem is being worked through. This problem had not been previously observed and so its occurrence and persistence have been quite perplexing, but we're optimistic we'll address the issue promptly. If you believe you have been affected with this issue (most users have not been), please email us at hello@engage.co so we can monitor your account. Thank you. On 7.7.16 this problem continued to be "in-process" with a fix. The engineers developed an alternative method to detect and confirm status between the dashboard and the status servers. A solution is currently in QA testing. An unscheduled maintenance window will occur after hours to reboot one or more servers and a software patch will be applied. To those affected, thank you for your patience. Many of you have taken time to help provide us more information and a fix wouldn't be possible without your cooperation, so thank you! On 7.8.16 a server reboot occurred, and a software patch was applied. We applied a new status check methodology which allows Engage to check both the dashboard AND status servers for availability. For those who are NOT double-confirmed as available for chat, you will be default timed-out in 90-120 seconds. We believe with this fix the issue has been resolved. If you continue to remain in an "online" status and don't believe you should be, please login to Engage, then logout. This will reset your status and going forward we will double-check your true availability.

Engage Error

• On Thursday May 30th we were alerted by a client about broken chat sessions occurring on an Android device. Engineering found the issue with Android. Chrome on android was recently updated, and the code that creates the desktop notifications was recognizing the Notification API on mobile, but when it tried to create the notification it failed and blocked incoming messages from being processed. As a result, it was possible for an agent in dashboard to not see messages coming in from a customer, or the ability for the customer to see incoming messages. The fix was tested and deployed on 6.1.16. In addition, engineering has added additional checks designed to keep this from happening in the future.

Engage Error

• At approximately 10:45am PDT on May 23rd, 2016 an updated version of Engage was pushed out that included new code that deployed to a single chat server and not the entire infrastructure, as intended. As a result, it was possible that for some users incoming chats were immediately disconnected and that actual and visible agent status could not be uniform. A key symptom of this issue was that a chat agent would be logged out of the dashboard, requiring them to login again. A fix was released at approximately 11:57am PDT. Steps have been taken to prevent this in the future and we apologize for any inconvenience this may have caused. Rest assured this was not infrastructure downtime, but rather a software release issue.

Engage Dashboard Changes pushed

• Administrators can now control (limit) the number of agents who appear in the widget. In some instances (especially where you have more than 30 agents "online" simultaneously) the number of available agents could be overwhelming to visitors. With this new setting you can limit the visible agents to a number (limit) you determine, such as 15 agents.

Engage Dashboard Changes pushed

• Administrators can now disable the widget on your website with a single click. This feature was designed so that when installing Engage and getting team members (users) onboarded the widget won't load. Once you're ready for showtime, you can click it and the widget will be live. Now you can install the widget code right away, but activate the widget when it makes sense for you.

Engage Dashboard Changes pushed

• Administrators can now change your team members status between "active" and "inactive" and also delete users. The inactive status means that the user is effectively hidden from displaying in widgets. This is a useful tool if an employee is gone on vacation or leave, or if they use Engage but are not part of the customer-facing set of team members you wish to display (such as IT people, marketing people, or admins). To change a user status: 1) login to the admin dashboard, 2) click on "edit account" then click on "team" and then choose the user and click on "status" where you'll be able to toggle between "active" and "inactive".

Engage Dashboard Changes pushed

• Administrators can now change your company directory domain name. When the domain has been changed, it is updated at the sub-directory structure in the URL. In addition, this change is automatically synced with the Engage API and updates all widgets and pointers without any required modifications.

• New reset password function.

Engage Dashboard Changes pushed

• From the Engage dashboard you can now edit your profile

• The Engage dashboard design and layout received modest changes

• The message dialogue window has messages of alternating colors

• Account admins can now edit user profiles

• Account admins can now change user passwords

• Account admins can now edit their company profile

• Account admins can now build unlimited widgets

• Users and admins can now add photos and logos

Features, Capabilities & Improvements in Development

• Syndication code Browser Preview Tracking which allows the dashboard to display any third-party site where you are syndicated to as a referrer source for incoming Engagements

• Color palette for widgets

• Make "end chat" and "email transcript" more obvious in the dashboard

• Ability for admins to add categories

• Ability for admins to delete/suspend

• Advanced statistics / analytics

• WebRTC updates

• Real-time B2B/B2C intelligence (Extensions Panel)

• Metadata and Schema

• Chat bots and chat automation

• Pre-canned messages (aka operator scripting)

• In-dialogue preview of shared URLs

• Design updates to HTML5 Profile page, including new "End Chat" button

• Adding "online" menu option to HTML5 Company Directory pages

• Flag to proactively show the same agent previously interacted with based on a tracking cookie

• Android / Google Play app, with push notifications

• Apple / iPhone app, with push notifications

• In-dialogue file/asset sharing

• More CRM integrations planned, including SugarCRM, Highrise, Insightly, Solve, PipelineDeals, SuiteCRM, vTiger, Zoho, baseCRM, Netsuite, Nimble, SAP and Infusionsoft

• Simplifying access to existing CRM integrations including Salesforce and Microsoft Dynamics

• Adding an "SMS" option for sending asynchronous client communications from the dashboard directly to the customers mobile phone