tag:status.airbase.com,2005:/historyAirbase Status - Incident History2024-03-28T05:25:19-07:00Airbasetag:status.airbase.com,2005:Incident/189716392023-10-30T13:52:25-07:002023-10-30T13:52:25-07:00Airbase dashboard is unavailable<p><small>Oct <var data-var='date'>30</var>, <var data-var='time'>13:52</var> PDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Oct <var data-var='date'>30</var>, <var data-var='time'>13:35</var> PDT</small><br><strong>Monitoring</strong> - A fix has been implemented by our upstream provider. We're continuing to monitor the results.</p><p><small>Oct <var data-var='date'>30</var>, <var data-var='time'>13:10</var> PDT</small><br><strong>Identified</strong> - Our upstream DNS provider is currently experiencing an outage, thus Airbase dashboard is unavailable for some customers. We will continue to post additional updates shortly.<br /><br />https://www.cloudflarestatus.com/incidents/l6x2h1zp69bc</p>tag:status.airbase.com,2005:Incident/186692432023-10-02T12:03:57-07:002023-10-03T21:34:43-07:00Airbase Dashboard is unavailable<p><small>Oct <var data-var='date'> 2</var>, <var data-var='time'>12:03</var> PDT</small><br><strong>Resolved</strong> - This incident has been resolved. It was caused by an incident at Cloudflare (https://www.cloudflarestatus.com/incidents/t53r6tfsg1sj), an infrastructure provider used by Airbase.</p><p><small>Oct <var data-var='date'> 2</var>, <var data-var='time'>11:43</var> PDT</small><br><strong>Update</strong> - We are continuing to monitor for any further issues.</p><p><small>Oct <var data-var='date'> 2</var>, <var data-var='time'>11:37</var> PDT</small><br><strong>Monitoring</strong> - We have implemented a fix, and the Airbase Dashboard is working as expected. We will continue to monitor this issue and provide additional updates as needed.</p><p><small>Oct <var data-var='date'> 2</var>, <var data-var='time'>11:27</var> PDT</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Oct <var data-var='date'> 2</var>, <var data-var='time'>10:58</var> PDT</small><br><strong>Investigating</strong> - Our engineering team is investigating reports that the Airbase Dashboard is unavailable. We will continue to post additional updates shortly</p>tag:status.airbase.com,2005:Incident/183286142023-08-30T19:50:17-07:002023-08-30T19:56:42-07:00Issues with outgoing email notifications<p><small>Aug <var data-var='date'>30</var>, <var data-var='time'>19:50</var> PDT</small><br><strong>Resolved</strong> - The issue has now been resolved.</p><p><small>Aug <var data-var='date'>30</var>, <var data-var='time'>17:00</var> PDT</small><br><strong>Identified</strong> - Our engineers have identified an issue with our downstream email notification provider. Some of our outgoing emails are currently not being delivered. We continue to work with them to resolve the issue.</p>tag:status.airbase.com,2005:Incident/175475502023-06-12T11:36:24-07:002023-06-12T22:54:35-07:00Issues with creating SVB cards<p><small>Jun <var data-var='date'>12</var>, <var data-var='time'>11:36</var> PDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jun <var data-var='date'>12</var>, <var data-var='time'>10:47</var> PDT</small><br><strong>Update</strong> - Card creation is working now. Issues still exist with viewing a card's available balance and editing the spend limit.</p><p><small>Jun <var data-var='date'>12</var>, <var data-var='time'>07:13</var> PDT</small><br><strong>Update</strong> - We received confirmation from SVB team that this issue is related to an ongoing incident with Mastercard.</p><p><small>Jun <var data-var='date'>12</var>, <var data-var='time'>06:26</var> PDT</small><br><strong>Update</strong> - We are continuing to work with SVB on a fix for this issue.</p><p><small>Jun <var data-var='date'>12</var>, <var data-var='time'>06:00</var> PDT</small><br><strong>Identified</strong> - Customers using SVB cards are unable to create new virtual cards from Airbase dashboard. We are working with SVB to get this resolved.</p>tag:status.airbase.com,2005:Incident/169432732023-04-23T01:24:56-07:002023-04-23T01:24:56-07:00Scheduled maintenance<p><small>Apr <var data-var='date'>23</var>, <var data-var='time'>01:24</var> PDT</small><br><strong>Completed</strong> - The scheduled maintenance has completed. Airbase web app and mobile app is now accessible.</p><p><small>Apr <var data-var='date'>23</var>, <var data-var='time'>01:00</var> PDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Apr <var data-var='date'>23</var>, <var data-var='time'>00:43</var> PDT</small><br><strong>Scheduled</strong> - Airbase will be down for scheduled maintenance for approximately 1 hour. We will use this time to improve the scalability and performance of the Airbase platform.<br /><br />During the short maintenance window, Airbase will not be accessible via web or mobile. We appreciate your patience and understanding.</p>tag:status.airbase.com,2005:Incident/165467202023-03-17T10:06:44-07:002023-03-17T10:18:27-07:00Users are unable to log in via SSO like Okta<p><small>Mar <var data-var='date'>17</var>, <var data-var='time'>10:06</var> PDT</small><br><strong>Resolved</strong> - Our downstream provider has fixed the issue and SSO-based login should work now.</p><p><small>Mar <var data-var='date'>17</var>, <var data-var='time'>09:56</var> PDT</small><br><strong>Identified</strong> - Customers who use SSO-based login like Okta are currently unable to log in because of a breaking change by a downstream provider. Our team is working on a fix. <br /><br />Users who are already logged in using SSO can continue with their session.</p>tag:status.airbase.com,2005:Incident/164464952023-03-16T11:45:44-07:002023-03-16T11:45:44-07:00SVB integration is currently impacted<p><small>Mar <var data-var='date'>16</var>, <var data-var='time'>11:45</var> PDT</small><br><strong>Resolved</strong> - Payments are fully operational. We will continue to work closely with customers to help complete any pending or on hold payments.</p><p><small>Mar <var data-var='date'>15</var>, <var data-var='time'>16:58</var> PDT</small><br><strong>Update</strong> - Payments have resumed now that the Silicon Valley Bank (SVB) platform is back online and testing is complete. This applies to bill payments (domestic and international), check payments, and reimbursement payments.<br /><br />An email will be sent to Airbase administrators today with details on how to complete or cancel in-flight transactions of various types and under different scenarios.</p><p><small>Mar <var data-var='date'>15</var>, <var data-var='time'>12:01</var> PDT</small><br><strong>Update</strong> - The Silicon Valley Bank (SVB) platform that we use to process bill payments, reimbursement payments, and checks are back online and we have successfully tested their platform to ensure services work as expected and plan to start to resume payments later today. This applies to bill payments (domestic and international), check payments, and reimbursement payments.<br /><br />Administrators will receive an email with further guidance on next steps later today.</p><p><small>Mar <var data-var='date'>14</var>, <var data-var='time'>14:06</var> PDT</small><br><strong>Update</strong> - The Silicon Valley Bank (SVB) platform started coming back online as of yesterday (Monday, March 13). Some payment information has started to flow from SVB to Airbase. Our priority now is on making sure that the services work as expected.<br /><br />At this time, if our testing determines that money can be moved safely and reliably, we plan to resume making payments on Wednesday, March 15. This applies to bill payments (domestic and international), check payments, and reimbursement payments.<br /><br />You can start queuing new payments. Anything scheduled from Monday, March 13 onwards will be actioned as normal from Wednesday, March 15, once we have confirmation that things are working as expected with SVB. Please refer to emails from Airbase sent to administrators that describe what happens next with in-flight payments that were initiated between Monday, March 6 and Sunday, March 12.<br /><br />We are in regular email contact with all Airbase administrators and will continue to provide updates as we receive them.</p><p><small>Mar <var data-var='date'>11</var>, <var data-var='time'>11:21</var> PST</small><br><strong>Monitoring</strong> - Due to the recent developments with SVB, there seems to be a complete outage of their API platform. As a result, Airbase customers who use the integration for SVB corporate cards and customers who use bill payments and reimbursements are currently impacted. SVB-issued virtual card creation, loading of transaction lists, and transaction processing are all impacted and currently failing. We are also not receiving any updates related to Physical card transactions. <br /><br />We currently don't have any updates on when the SVB platform will be operational again and will keep this page updated as we learn more.</p>tag:status.airbase.com,2005:Incident/163118862023-02-28T22:10:21-08:002023-02-28T23:02:37-08:00Airbase Dashboard Unavailable<p><small>Feb <var data-var='date'>28</var>, <var data-var='time'>22:10</var> PST</small><br><strong>Resolved</strong> - Airbase dashboard was unavailable for 15 mins between 05:54 and 06:08 UTC ( 21:54 and 22:08 PST) due to unplanned maintenance</p>tag:status.airbase.com,2005:Incident/162964382023-02-27T11:59:27-08:002023-02-27T11:59:27-08:00Issues with creating and viewing SVB and Amex Cards.<p><small>Feb <var data-var='date'>27</var>, <var data-var='time'>11:59</var> PST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Feb <var data-var='date'>27</var>, <var data-var='time'>11:18</var> PST</small><br><strong>Update</strong> - Issue is mitigated now. We are continuing to monitor for any further issues.</p><p><small>Feb <var data-var='date'>27</var>, <var data-var='time'>11:08</var> PST</small><br><strong>Monitoring</strong> - A fix has been implemented by our partner team. We will continue monitoring our systems</p><p><small>Feb <var data-var='date'>27</var>, <var data-var='time'>10:43</var> PST</small><br><strong>Update</strong> - We are continuing to work with our partner on a fix for this issue.</p><p><small>Feb <var data-var='date'>27</var>, <var data-var='time'>10:21</var> PST</small><br><strong>Identified</strong> - Customers on Amex/SVB card programs are unable to view or create corporate cards. This is caused due to a dependent service outage. Our partner is working to fix this with the highest priority.</p>tag:status.airbase.com,2005:Incident/157652212023-01-03T12:37:38-08:002023-01-03T12:37:38-08:00Airbase dashboard outage<p><small>Jan <var data-var='date'> 3</var>, <var data-var='time'>12:37</var> PST</small><br><strong>Resolved</strong> - The incident has been resolved.</p><p><small>Jan <var data-var='date'> 3</var>, <var data-var='time'>10:00</var> PST</small><br><strong>Monitoring</strong> - A temporary fix has been implemented. We are continuing to monitor the issue.</p><p><small>Jan <var data-var='date'> 3</var>, <var data-var='time'>09:41</var> PST</small><br><strong>Identified</strong> - Some customers have reported experiencing issues accessing the dashboard. Our team is currently investigating.</p>tag:status.airbase.com,2005:Incident/136554262022-11-22T17:42:55-08:002022-11-22T17:42:55-08:00Airbase dashboard outage<p><small>Nov <var data-var='date'>22</var>, <var data-var='time'>17:42</var> PST</small><br><strong>Resolved</strong> - The incident has been resolved. We continue to monitor our systems for any residual issues or regressions.</p><p><small>Nov <var data-var='date'>22</var>, <var data-var='time'>17:23</var> PST</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Nov <var data-var='date'>22</var>, <var data-var='time'>15:53</var> PST</small><br><strong>Update</strong> - Some of the features/pages are still facing an issue. We are still working on a fix.</p><p><small>Nov <var data-var='date'>22</var>, <var data-var='time'>14:36</var> PST</small><br><strong>Update</strong> - We are continuing to work on a fix. Majority of the dashboard pages should be working normally now. A few parts of the website are still facing degraded performance.</p><p><small>Nov <var data-var='date'>22</var>, <var data-var='time'>14:21</var> PST</small><br><strong>Identified</strong> - We've identified the issue and are working on a fix.</p><p><small>Nov <var data-var='date'>22</var>, <var data-var='time'>13:31</var> PST</small><br><strong>Investigating</strong> - Some customers have reported experiencing issues accessing the dashboard. Our team is currently investigating.</p>tag:status.airbase.com,2005:Incident/126194832022-10-25T21:17:52-07:002022-10-25T21:17:52-07:00Dashboard experiencing degraded performance<p><small>Oct <var data-var='date'>25</var>, <var data-var='time'>21:17</var> PDT</small><br><strong>Resolved</strong> - The incident has been resolved. All systems are back to normal and some upgraded ones better than before :-)</p><p><small>Oct <var data-var='date'>25</var>, <var data-var='time'>21:00</var> PDT</small><br><strong>Monitoring</strong> - During a routine infra upgrade, we encountered unforeseen issues with one of our downstream providers. This partially affected the performance and availability of the dashboard for a portion of the users.<br />The issue has now been fixed and all our monitoring shows all systems are back up. We are still continuing to monitor it.<br />Card transactions were not affected by this issue.</p>tag:status.airbase.com,2005:Incident/111477932022-09-16T04:38:59-07:002022-09-29T10:27:56-07:00Issues with Dashboard access<p><small>Sep <var data-var='date'>16</var>, <var data-var='time'>04:38</var> PDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Sep <var data-var='date'>15</var>, <var data-var='time'>21:30</var> PDT</small><br><strong>Monitoring</strong> - A fix has been implemented and the dashboard is operational.</p><p><small>Sep <var data-var='date'>15</var>, <var data-var='time'>20:55</var> PDT</small><br><strong>Update</strong> - We are continuing to investigate this issue with dashboard access. We determined card transactions are not impacted by this incident.</p><p><small>Sep <var data-var='date'>15</var>, <var data-var='time'>19:28</var> PDT</small><br><strong>Investigating</strong> - Airbase services are unavailable. Our engineers are investigating the issue. We will post more updates as we learn more.</p>tag:status.airbase.com,2005:Incident/108961812022-08-23T18:45:25-07:002022-08-23T20:14:05-07:00Issues with upstream DNS provider<p><small>Aug <var data-var='date'>23</var>, <var data-var='time'>18:45</var> PDT</small><br><strong>Resolved</strong> - The incident has been resolved.</p><p><small>Aug <var data-var='date'>23</var>, <var data-var='time'>16:50</var> PDT</small><br><strong>Identified</strong> - We are investigating an issue with an upstream DNS provider. The dashboard is inaccessible for some users. We are currently working to fix the issue.</p>tag:status.airbase.com,2005:Incident/105959782022-07-15T13:53:57-07:002022-07-15T13:53:57-07:00Dashboard pages automatically refreshing<p><small>Jul <var data-var='date'>15</var>, <var data-var='time'>13:53</var> PDT</small><br><strong>Resolved</strong> - A fix has been deployed. We will continue to monitor the issue.</p><p><small>Jul <var data-var='date'>15</var>, <var data-var='time'>13:11</var> PDT</small><br><strong>Identified</strong> - We have identified an issue with one of our third party providers causing redundant browser refreshes. We will be issuing a fix shortly.</p><p><small>Jul <var data-var='date'>15</var>, <var data-var='time'>12:57</var> PDT</small><br><strong>Investigating</strong> - We are currently investigating reports that some pages on the Airbase dashboard are automatically being refreshed in the browser. We will update further soon once we've identified the cause.</p>tag:status.airbase.com,2005:Incident/105065682022-07-09T04:48:58-07:002022-07-09T04:48:58-07:00Scheduled database upgrade<p><small>Jul <var data-var='date'> 9</var>, <var data-var='time'>04:48</var> PDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Jul <var data-var='date'> 9</var>, <var data-var='time'>04:28</var> PDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jul <var data-var='date'> 8</var>, <var data-var='time'>13:35</var> PDT</small><br><strong>Scheduled</strong> - We will be undergoing a scheduled maintenance on Saturday. We are adding resources and upgrading our database to make Airbase more reliable and performant. The dashboard will be inaccessible during the upgrade, but the card transactions are expected to function normally.</p>tag:status.airbase.com,2005:Incident/101598422022-05-26T06:00:00-07:002022-06-06T07:25:09-07:00Degraded performance of the Airbase web app<p><small>May <var data-var='date'>26</var>, <var data-var='time'>06:00</var> PDT</small><br><strong>Resolved</strong> - Some pages on the Airbase web app (listing pending requests, ...) suffered degraded performance. Some users may have seen delays in syncing ledger entries to the GL and other background tasks. The incident lasted approximately 90 minutes (from 06:00 Pacific) before being resolved.</p>tag:status.airbase.com,2005:Incident/100221602022-05-17T10:00:00-07:002022-05-18T10:32:35-07:00Airbase dashboard outage<p><small>May <var data-var='date'>17</var>, <var data-var='time'>10:00</var> PDT</small><br><strong>Resolved</strong> - The Airbase web application was unavailable for approximately 45 minutes starting around 10:20am Pacific and suffered degraded performance for a further 30 minutes. Airbase cards continued to work during this time. The outage was identified and resolved.</p>tag:status.airbase.com,2005:Incident/98552992022-04-26T01:16:34-07:002022-04-26T01:16:34-07:00Infrastructure issues with upstream provider<p><small>Apr <var data-var='date'>26</var>, <var data-var='time'>01:16</var> PDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Apr <var data-var='date'>25</var>, <var data-var='time'>23:36</var> PDT</small><br><strong>Identified</strong> - We're seeing issues with one of our cloud infrastructure providers, which may impact some functionality on the Airbase dashboard.<br />We are currently investigating and will report back once we know more.</p>tag:status.airbase.com,2005:Incident/94002312022-02-24T10:08:52-08:002022-02-24T10:08:52-08:00Airbase dashboard is unreachable<p><small>Feb <var data-var='date'>24</var>, <var data-var='time'>10:08</var> PST</small><br><strong>Resolved</strong> - This incident has been resolved with our cloud provider.</p><p><small>Feb <var data-var='date'>24</var>, <var data-var='time'>09:06</var> PST</small><br><strong>Update</strong> - We are continuing to monitor for any further issues.</p><p><small>Feb <var data-var='date'>24</var>, <var data-var='time'>08:51</var> PST</small><br><strong>Monitoring</strong> - The service disruption that occurred was due to an outage with our upstream cloud provider. The service seems to have been restored. We continue to monitor it.</p><p><small>Feb <var data-var='date'>24</var>, <var data-var='time'>08:47</var> PST</small><br><strong>Investigating</strong> - We are currently investigating the issue.</p>tag:status.airbase.com,2005:Incident/93279442022-02-15T16:48:17-08:002022-02-15T16:48:17-08:00Connectivity issues while loading dashboard<p><small>Feb <var data-var='date'>15</var>, <var data-var='time'>16:48</var> PST</small><br><strong>Resolved</strong> - The incident is now resolved by our DNS provider. Both internal users and customers have reported being able to access dashboard successfully.</p><p><small>Feb <var data-var='date'>15</var>, <var data-var='time'>16:00</var> PST</small><br><strong>Monitoring</strong> - Some customers have reported facing difficulties in accessing our dashboard. Our upstream DNS provider Cloudflare has been experiencing network performance issues, which caused this incident. The provider has reported that the problem has been resolved, but we are continuing to monitor the result.</p>tag:status.airbase.com,2005:Incident/91743572022-01-25T01:30:00-08:002022-01-27T00:08:30-08:00Failures while loading web dashboard<p><small>Jan <var data-var='date'>25</var>, <var data-var='time'>01:30</var> PST</small><br><strong>Resolved</strong> - A release to the frontend dashboard caused pages to go in a loop affecting a subset of users with the certain features and privileges. These users were unable to load the site during this time. Our alerting systems caught the anomaly and we rolled back the change quickly.<br />We are trying to mitigate such incidents by ramping up our integration testing efforts, with the goal of testing all features against all possible user roles before every release.<br /><br />Payment transactions and other functions within Airbase were not affected by this.</p>tag:status.airbase.com,2005:Incident/87867622021-12-08T04:30:12-08:002021-12-08T04:30:12-08:00Availability issues with upstream provider<p><small>Dec <var data-var='date'> 8</var>, <var data-var='time'>04:30</var> PST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Dec <var data-var='date'> 7</var>, <var data-var='time'>14:42</var> PST</small><br><strong>Update</strong> - All systems are back to operational. We are still continuing to monitor the status with our platform provider.</p><p><small>Dec <var data-var='date'> 7</var>, <var data-var='time'>14:39</var> PST</small><br><strong>Update</strong> - There are some early signs of recovery. We continue to monitor the issue.</p><p><small>Dec <var data-var='date'> 7</var>, <var data-var='time'>14:33</var> PST</small><br><strong>Monitoring</strong> - Our upstream platform provider is experiencing an outage. Access to the dashboard might be impacted for some customers. We are actively working on mitigation paths for this issue.</p>tag:status.airbase.com,2005:Incident/82837582021-10-21T11:57:07-07:002021-10-21T11:57:07-07:00Virtual card details not loading<p><small>Oct <var data-var='date'>21</var>, <var data-var='time'>11:57</var> PDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Oct <var data-var='date'>21</var>, <var data-var='time'>10:18</var> PDT</small><br><strong>Identified</strong> - The ability to view and copy your virtual card details is currently unavailable. This is due to one of our card providers experiencing an outage and we are working with them to resolve the issue as soon as possible. Transactions and other functions within Airbase have not been impacted.</p>tag:status.airbase.com,2005:Incident/80851012021-09-26T06:43:29-07:002021-09-26T06:43:29-07:00Database maintenance in progress<p><small>Sep <var data-var='date'>26</var>, <var data-var='time'>06:43</var> PDT</small><br><strong>Resolved</strong> - The issue has been resolved. We're continuing to monitor it.</p><p><small>Sep <var data-var='date'>26</var>, <var data-var='time'>06:23</var> PDT</small><br><strong>Update</strong> - There is an issue with our Cloud provider. We're currently working to resolve it.</p><p><small>Sep <var data-var='date'>26</var>, <var data-var='time'>05:29</var> PDT</small><br><strong>Monitoring</strong> - We're performing routine database maintenance and upgrade. Airbase Dashboard will be unavailable for the next 5-10 mins.<br />All transactions will go through as normal.</p>