A description of RevSpring, Inc.’s data processing and document mailing services.
At each facility, RevSpring’s infrastructure is built with redundancy and availability in mind. Dual WAN (ISP) connections are available and ready for failover. Additionally, redundant firewalls are in use at each location to ensure the loss of one does not prevent RevSpring from operating as normal. All inter-switch connections utilize multiple links to different switches in each stack, ensuring that loss of any switch does not interrupt RevSpring’s ability to provide clients data services and solutions. The core data processing environment at RevSpring is a fully virtualized VMware Enterprise farm running on a dedicated storage array, supporting automatic failover in case of physical server loss and live migrations for maintenance and uptime stability. An additional Microsoft Failover Cluster with a dedicated SQL storage array is utilized for high availability of production data shares and database servers. Additional processing is provided via additional servers dedicated for individual production purposes. Access to these infrastructure systems is strictly limited to network operations and the IT support staff as necessary. Additional infrastructure is utilized to provide dedicated testing environments for the Application Development teams. This allows for completely isolated versions of the data processing environments for end to end production testing and quality assurance.
RevSpring’s production processes are driven by proprietary software developed in-house. This allows for the fastest turnaround for customized client requirements as well as ensures that all aspects of the platform can be optimized for performance and controlled for secure handling of data. Dedicated in-house development teams focus on producing improvements and client-requested changes to the production processes, ensuring high quality coding standards are followed. Enterprise security applications are used to ensure that confidentiality, integrity, and availability of client data is protected across the entire environment. Vormetric Encryption Expert is used for encrypting data at rest, while SFTP software is used for data in transit. Additionally, LogRhythm is used to provide a centralized, fully reportable database of all security and application event logs on critical systems. Symantec Endpoint Protection is used across every server and workstation in the environment to ensure that hosts are protected from viruses and malware. Additionally, all internet endpoints a protected by firewalls with active Content Filtering, Gateway Anti-Virus, and Intrusion Prevention Systems, minimizing the risk of internet sources introducing malicious code into the environment.
RevSpring’s personnel is broken into several teams which support different business functions of the company. The Finance, Human Resources, Sales, and Marketing teams are responsible for the standard business functions that these teams are responsible for in most companies. Additionally, there is an IT team and an Operations team that handle the direct production data and processing functions for the company. The IT Team contains the Account Management teams, who are responsible for handling all client requests, tracking them throughout their execution, and notifying clients of impacts to their data’s processing. The IT Team also contains the IT Operations teams, who handle the internal support of workstations and infrastructure, as well as the monitoring of the production software applications to ensure that data processing is not interrupted. Finally, the IT Team contains the Application Development teams, who focus on producing the changes requested by clients for their data processing, as well as improving current products and designing and implementing new products and services. The Operations Team contains management to oversee all the different aspects of the physical output of production data. It also contains the base Print teams, responsible for generating the physical letters once submitted to production from the data processing platforms. Insert teams are responsible for taking the printed outputs and matching them with the appropriate envelopes and any additional inserts, getting them all in the letters and sealed, ready to mail. Throughout Operations, the Quality teams have provided spot checks and quality reviews to ensure that the produced output is high quality and on time.
Data Acceptance and Validation
RevSpring accepts data via Secure FTP (FTP over SSH) from its clients. All files are compared against recently received files to determine if the file might be a duplicate of a previously recently submitted file. Files flagged as potential duplicates are placed to the side for manual review by the Account Management team. Additional checks, such as validating file completeness, are done based on the supported features of the data file format used by the client. End of file markers, record count checks, CRC checksums, and other methods are used. Any file that supports these completeness checks and fails them is rejected out of the system automatically and the client notified of the failure. Once all checks are complete and the file is accepted into production, a history of the file is recorded so that it can be used to check new files for duplicates. Test files are also accepted through our production platform and SFTP site. These files are identified by the client either naming the file with the word “TEST” in the file name, or else by being uploaded to a dedicated “TEST” folder instead. If the client uploads test data, it is pulled out of the production processing stream prior to any checks, and Account Management is notified that test data has arrived for the client.
Once a file is accepted for processing, numerous data services are ran against the submitted data. The services ran on specific data is dependent on what the client has chosen to run on their data. The data services include, but are not necessarily limited to:
- Address Standardization
- National Change of Address Lookup
- Bankruptcy Lookups
- Deceased Lookups
- Phone Number Lookups
- Address Element Corrections
- Litigious Debtor Scrubbing
- Charity Scoring
Additional services can be determined at time of contract or upon a new service request from the client. Any data returned from these services is provided back to the client via SFTP, the client’s SFTP site, or other means as approved by the client. In many cases, these data services are directly integrated into the client’s collection software platform, allowing for seamless ingestion and processing of the return data with minimal to no client interaction required.
Once all primary data services are completed, the communications processing functions begin. These involve generating letters, creating electronic mail notifications, bill presentment documents, and other requested items based on the data provided. Physical letters are printed, inserted and sent to the USPS mail stream during normal business hours. Electronic communications are delivered during agreed upon hours with the client, ensuring that electronic communications can comply with other collection communication laws and regulations. Bill presentment / payments are provided on our secure processing portals, allowing end recipients to quickly and easily access their recent bills, provide payments, and contact the clients for any other questions. After communications are generated and sent, several additional data services are available. These include mail tracking, USPS based address services, and intelligence and analytics services. These services provide an additional layer of visibility into the communications process for the client, ensuring that they are receiving the best solution set for their business operations.
The production processing platform at RevSpring’s Oaks, PA facility is highly automated, with regular production times scheduled for Monday – Thursday 24hrs, and Friday, Saturday, and Sunday from 6am to 6pm. During these times, a dedicated staff is responsible for monitoring the applications and services for any issues. Error handling is regularly addressed within minutes of occurring, and any issue that needs to be escalated to the client will be directed to Account Management for immediate communication to the client. During off-hours, file acceptance, data services, and post-communication services continue to run. An on-call schedule is utilized to ensure that there is 24×7 coverage of any issues related to production processing. Critical issues with files after-hours are escalated to Account Management to be communicated to the client as soon as possible.
The production processing platform at RevSpring’s Phoenix, AZ facility is highly automated, with regular production times ranging from 9am to 5pm Monday through Friday. During these times, a dedicated staff is responsible for monitoring the applications and services for any issues. Error handling is regularly addressed within minutes of occurring, and any issue that needs to be escalated to the client will be directed to Account Management for immediate communication to the client. During off-hours, file acceptance, data services, and post-communication services continue to run. An on-call schedule is utilized to ensure that there is 24×7 coverage of any issues related to production processing. Critical issues with files after-hours are escalated to Account Management to be communicated to the client as soon as possible.
Physical access to RevSpring facilities is controlled through the use of a personal security access card system. Access privileges are determined based upon department and are approved by appropriate management personnel. The computers that manage the badge reader systems are housed in physically secured rooms and are protected by password controls. Semi-annually, management reviews badge reader system reports listing employees and their assigned physical access to RevSpring facilities in order to ensure access granted to all employees is appropriate.
RevSpring’s data operations areas are separated from the other business function areas with access granted to Information Technology (IT), Executive or employees granted access by CIO only. The datacenters are equipped with security cameras. Servers are housed in the datacenters with a restricted security access card system and video surveillance. Consultants and third-party technicians are monitored constantly while performing work in any restricted access area.
The main entrances are monitored by receptionists, who greet visitors. Visitors are required to provide picture identification and are entered in our visitor log. Each visitor is issued a visitor badge, and is escorted and monitored at all times during the visit. Intrusion detection is monitored 24/7 by an alarmed security system and surveillance cameras located throughout the facilities. Security camera activity is stored on hard drives for up to 90 days and is reviewed on an as-needed basis.
The data center facilities are controlled for fire, temperature, humidity, and power failure by automated systems. RevSpring’s datacenters and production facilities are equipped with smoke detectors and a modern, overhead sprinkler system for fire protection. The datacenters are equipped with an FM-based fire suppression system. The temperature of the computer room is controlled by dual dedicated air conditioning systems which are inspected and tested for functionality on a regular basis. In the event of a power outage, RevSpring’s electrical generators would prevent interruption to scheduled production. This generator is load tested bi-weekly for a half hour.
Access to RevSpring’s network operating systems is granted to employees based on their department and position. Production personnel do not require logical access. Upon office personnel’s contractual signature to begin employment with RevSpring, departmental managers fill out an electronic request via RevSpring’s Intranet for New Hire Setup. Network Administrator sets up a new user name and strong password controls (at least six characters, and at least three of the following four groups: uppercase, lowercase, numerals, and non-alphabetic) for the employee and emails the information to the departmental manager. The first time the employee logs into the system they are required to change their password. Employees are required to change their passwords every 45 days.
To complement the system password setting, other security configuration settings exist to protect against unauthorized access to systems. After a period of inactivity, users are locked out until the password is re-entered. Accounts are locked out after five invalid login attempts and are automatically unlocked after 15 minutes.
Network, LAN/WAN, firewall and server security is maintained and monitored through system audit logs where transactions are tracked at the username level. Different logs include FTP, systems, and applications. A complete history of information security administration actions and system operator actions is maintained by username for 90 days. A complete history of support personnel and manager actions is maintained by username indefinitely. Audit trails contain date and time of each event, user ID, type of event, type of access, terminal, port, and location information. Firewall security is monitored internally by RevSpring.
Access to the network applications, programs, and databases is restricted to Network Administrators. Access to modify programs and/data files remotely is restricted to Network Administrators and IT Managers. Some clients are granted limited access to strictly controlled custom web-based functions. Certificates are used for security, authentication and encryption of the browser connections. All activity is logged and the connection is secured via SSL (Secure Sockets Layer).
Annually, department managers review a listing of all employees in their department and their corresponding system capabilities and recertify the adequacy of their system access levels.
Semi-annually, IT management reviews a list of administrators and their corresponding access to application programs and data directories, databases, and application configuration and recertifies the adequacy of their access levels.
When employees are terminated, human resources generates an employee separation checklist in the system. The network operations team automatically receives an email stating termination date and employee name and department. User accounts are disabled immediately and are removed after three months.
Remote access means any access to the network through a non-RevSpring controlled network device or medium. Employees, contractors, vendors, and agents with remote access privileges to the network are required to ensure that their remote access connection is given the same consideration as the user’s on-site connection to the network. Remote access is managed using a VPN connection which is utilizes the strong encryption and two-factor authentication, with permissions managed via Active Directory Security groups. All user accounts must be authenticated with RevSpring’s Active Directory domain.
Employees who use a RevSpring provided computer are allowed access to a Windows SSTP VPN, which allows them to operate as if they were on-site with their machine. All non-RevSpring provided workstations only have access to a web-based Remote Desktop solution, providing SSL VPN support while only allowing that computer Remote Desktop access to authorized systems within the network.
All system changes, whether to internally programmed custom applications, infrastructure, web front-ends, or databases, comply with RevSpring, Inc.’s Change Management Policy. This provides a high level of control and assurance that changes made not only properly addressed the stated objectives, but that they also introduced no additional security, integrity, or production availability risks.
All changes are documented, including design, deployment steps, risk levels, potential specific risks, and rollback instructions. These changes are reviewed, and if approved, executed into the production environments as appropriate. Additional measures for internally developed application changes include a dedicated code review, used to identify common errors in coding practices and security concerns; quality assurance testing, used to verify that the changes have the desired effect and no more; and test results review, used to ensure adequate testing was completed prior to production deployment.
All changes are reviewed and must be approved by the business stakeholder prior to deployment. These changes may be simple infrastructure changes or a complex new product rollout. Either way, the stakeholder determines if a dedicated risk meeting needs to be held to review the change prior to execution. If the risk is high, then the meeting identifies additional safety measures that should be included or can be done post-deployment to ensure all identified risks are mitigated. Deployment of approved changes is done by the appropriate system owners. Where possible, the developer / designer will also be present at the deployment to provide additional oversight to the deployment process. Once deployed, any changes to the deployment steps are noted on the change documentation, and the documents are archived for future reference. Any additional post-deployment checks and steps are initiated at that point and then production operations are resumed.
The RevSpring environment is not conducive to employ a true “emergency change” management system. If an emergency change is required, the emergency change procedures are exactly the same as the normal change procedures except that in some cases it may require post-deployment documentation. Risk analysis, design review, and deployment planning is still completed, but may need to be done “on-the-fly” to address an immediate issue. These changes are still required to be documented and reviewed post-deployment as a normal change is.
Disaster Recovery / Business Continuity
RevSpring utilizes internal redundancy and excess capacity wherever possible internally to ensure that nothing short of a major disaster can cause a complete failure to operate at any of its facilities. This includes redundant network communications, excess capacity on the production floors, isolated fire suppression, backup generators and environmental controls, and other methods to ensure that any facility can continue to process. However, to ensure business continuity, it also uses its alternative processing plants for backup/redundancy in case of work stoppage due to disaster in one of its facilities. RevSpring tests its disaster recovery procedures annually to ensure that in the event of a disaster, business operations can continue out of the other facilities and clients are impacted minimally. Wherever possible, RevSpring strives to provide consistent, predictable, and reliable services for its clients without requiring client intervention (changing of IPs, configuration changes, etc…) so that clients can continue to utilize RevSpring services without interruption.
Data as described by RevSpring constitutes the following:
- Client Reports – Processing reports, confirmations, return data and other system-generated files that are returned to the client to provide processing transparency and service data.
- Client files – Data files submitted to RevSpring for data processing and mail / communication generation.
- System files – Files generated by or controlling the production application and its services to ensure proper operation and functionality.
- Database records – SQL / MySQL based tracking and data archives used for providing additional services and oversight of the processing platform’s health and activities.
- Exception reports – Email and SMS based alerts identifying issues related to the processing environment and any data processes that need to be addressed.