AGILE  DEVELOPMENT  AND  AUTOMATED  SOFTWARE  TESTING

Here's my answers to two questions raised at one of my recent webinars on software / firmware V&V and documentation:

Ques 1:  Do you have experience working with Agile methodologies such as SCRUM? In your presentation, you mention that FDA suggests following a waterfall development cycle. Do you know what is the point of view of the FDA about iterative/incremental methodologies?

Ans 1:  My experience in Agile is limited, although, as mentioned its principles have been used in many companies before someone came up with the name Agile (as is true with many other "methodologies", e.g., 6 sigma). 

I showed the one slide to illustrate V&V (Verification and Validation), which showed a "waterfall" product development cycle.  It was used by the U.S. FDA, in the mid 90's and was focused on design control (21 CFR 820.30; see http://www.fda.gov/medicaldevices/deviceregulationandguidance/guidancedocuments/ucm070627.htm
-- Figure 1 under "III. APPLICATION OF DESIGN CONTROLS").  It was used in conjunction with a process that typically is iterative / incremental -- device R&D to illustrate V&V for design control (21 CFR 820.30), and was not meant to show any FDA preference in product or software development, just how a series of device verifications lead to a device validation.  The FDA has no preference as to what development methodologies a company selects and uses, that I have seen to date -- they leave such decisions to the manufacturer, who must justify and prove / document their choices.  However, the FDA wants the documentation to prove defined processes were followed and that there was compliance to the regulations.  Hence my caution re: Agile, which manifesto on the Internet states that a key Agile goal is an implied minimization of defined processes ... and a reduction in documentation, to wit:
...
"Individuals and interactions over processes and tools

"Working software over comprehensive documentation..."  

-- http://www.agilemanifesto.org 

So just proceed with that caution in mind.

Ques 2: You haven’t discussed automated tests (for unit tests, integration tests, functional tests, performance tests). Wouldn’t it be the perfect tool to demonstrate reproducibility of a system? What is the point of view of the FDA on automated tests?

Ans 2:  There's nothing wrong with automated testing.  They are as mentioned, an excellent tool.  Much is done in software / firmware program development using such.  However, the automated test programs and hardware must themselves be rigorously validated in the same way as the webinar discussed (including 21 CFR 11; and see "SOFTWARE / FIRMWARE  V&V  "MODEL"" post below) before their data can be used in subsequent V&V activities.  So a discussion of automated testing is redundant to the subject of software testing as  discussed (see previous blogs on the subject).   Be aware that the FDA would look very carefully at such automated test equipment and programs, and their V&V, since they are then used for subsequent automated testing for V&V of other software on a repetitive basis.

-- John E. Lincoln, jelincoln.com