IT:AD:Continuous Testing:HowTo:Test Pipeline
- See also:
Summary
The basic reality is that it will be prohibitively expensive, take too long and be ineffective trying to change mindsets when one can implement a gated checkin pipeline that ensures compliance right from the first checkin.
Common Pipeline Elements
Summary
As per Martin Fowler's description1) Continuous Integration (CI) is the practice of merging development work with a shared mainline branch frequently – at least daily – in order for each integration to be verified by an automated build and automated tests to ensure changes work and work with other changes.
It has been suggested2) that a more precise term would have been IT:AD:Continuous Build.
A core concept of IT:AD:Continuous Integration is that only a small fraction of new code is added – specific to a single work item – so the cause of a test failure is easier to find. In older build systems, faulty code is already merged with the primary trunk before the build server can test and fail the new work (leading to “Breaking the Build” scenarios). Current build servers can mitigate (see IT:AD:VSTS:VCS:Git:HowTo:Branch Policy).
The primary function is to test the changes, and report back to the developer. It is not to deploy the code to a target environment. A more mature paradigm is the concept of IT:AD:Continuous Delivery system.
- Automated penetration testing looking for Sql injection and other means in via web UIs:
- FxCop
- How to get it to run: http://stackoverflow.com/a/32093939
- Fortify:
- More: https:en.wikipedia.org/wiki/Listoftoolsforstaticcodeanalysis * Maintainabilty of code: * IT:AD:StyleCop * IT:AD:CodeIt.Right * IT:AD:Resharper (using the free CLI) * Compliance with Web UI Usablity and Accessibilty standards: * IT:AD:HTML Tidy * Total Validator * IT:AD:CSE HTML Validator * Performance: * Ants Performance Profiler * JetBrains Profile * JetBrains DotMemory * Coverage: * JetBrains DotCover * Sql code: * Sql Code GuardThat said, don't get complacent – automation can't do everything3). The main goal of build server validation is a constant reminder to be thinking right from the start about security, performance, maintainability, usability and accessibility with every check-in. That is the big win.
### Tool Comparison ^Tool^Notes^ |IT:AD:StyleCop|| |:::|Advantages:| |:::|* Free| |:::|* Open Source| |:::|* Can run on Hosted Build Agents.| |:::|Considerations:| |:::|* Surpassed by flexibility of IT:AD:CodeIt.Right| |:::|* Buggy in IT:AD:Visual Studio 14.| |:::|* May not be able to create custom rules.| |:::|* Pretty dated. May not be able to understand latest C# syntax.| |:::|Disadvantages:| |:::|* Rigidly adheres to Microsoft Standards, defined over a decade ago.| |IT:AD:CodeIt.Right|| |:::|Advantages:| |:::|* Greater flexibility than IT:AD:StyleCop| |:::|* Already has the same rules as IT:AD:StyleCop.| |:::|* Can be used to define custom rules.| |:::|Considerations:| |:::|* Enterprise licensing might be an issue.| |:::|Disadvantages:| |:::|* Not free.| |:::|* Not Open Source.| |IT:AD:Resharper|| |:::|Advantages:| |:::|* Very trusted by developer community.| |:::|* Focuses on what's needed.| |:::|* Free if using just the CLI in the Build Server| |:::|Considerations:| |:::|* Does not have as many rules as IT:AD:StyleCop.| |:::|Disadvantages:| |:::|* Not Open Source.| ## Resources ##