Key Findings

1. Care should be taken to ensure that all users should have an equal voice in the requirements gathering process, particularly when they represent sub-organizational units of varying size, funding, and clout.
2. Assessment of vendor size, stability, support, and responsiveness may be definitive in cases where alternatives are judged to be of rough technical equivalence.
3. Regulatory requirements may constrain an organization's ability to adopt new or alternative technologies.

Challenge

The Electronic Submissions Gateway (ESG) is a managed File Transfer (MFT) system that functions as the single point of entry for all electronic regulatory submissions, including New Drug Applications, Investigational New Drug Applications, Biologics License Applications, Master Files, and Adverse Event Reports for various centers at the FDA. The ESG has a security rating of FISMA High.

The ESG accepts encrypted submissions from external users ranging from large pharmaceutical companies to small medical offices. The submissions may range in size from a few thousand kilobytes to over one hundred gigabytes. Submissions received by the ESG are routed to the appropriate FDA center or office based on metadata included in the submission. The FDA contracted with TCG to document the current system architecture and evaluate the ability of existing and alternative solutions to handle an expected volume of 25 to 30 million submissions by 2020 with individual submissions ranging as high as a terabyte.

Solution

TCG developed a comprehensive Study Plan for approval by the client. We then documented the existing system ESG architecture and interviewed stakeholders to document current business and technical requirements. We used the interviews to create a list of user stories which became the basis for an updated functional requirements list. The final requirements were evaluated as critical or noncritical and used to populate the DAR. The DAR was evaluated and included in the major deliverable, an Analysis of Alternatives document that detailed the process, results, and recommendations. After FDA reviewed the AoA report, we produced a phased transition report that discussed the path to implementation of the TCG recommendations.

Results

TCG reviewed over 40 documents and interviewed 43 stakeholders primarily from FDA centers, but including representatives of industry and other end-users. We used the interviews to create a list of 125 user requirements and 76 summary requirements. These were validated with the interviewees and the OIMT COR and summarized as 34 final requirements for the DAR. We developed a set of candidate vendor offerings, derived from client discussions, computer industry reports, and Internet searches. We sent questionnaires based on the DAR to twelve companies offering qualified MFT products and received responses from eight.

The DAR was finalized with the 34 requirements arrayed across the 8 vendor solutions. The DAR allowed vendors to describe their product as fully, partially or not meeting a given requirement. We evaluated the responses and adjusted answers in cases where our research showed an answer that over or undervalued a given capability. A ranking of the solutions was produced based on the calculated score for each vendor. We delivered a final Analysis of Alternatives report documenting the entire process, including our results and underlying data, and discussing our recommendation for one of the three alternatives.

Finally, TCG wrote and delivered a phased transition report, suggesting processes to implement the transition to the recommended solution for the ESG. We delivered all reports on-time and within budget.