Let the community know what you think. Share your opinions now!
Experience automating different application types and staffing the project to be a success. You also need to ask if your SMEs have the capacity to help with test strategy. You need to first see what type of test you are trying to automate and how the applications are architected. For instance is it a thin-client, thick-client, hosted through a non-persistent VDI, mainframe, does it use unfriendly HTML tags (i.e. canvas), SAP, etc. You need to see if the company providing testing as a service has knowledge in this area. You need to make sure they are doing a test tool evaluation before picking an automation framework. Often times this may require multiple automation frameworks based on application types.
At the end of the day you need to see to it that the project is staffed correctly. You need to realize that Automation isn't some "easy" button that allows you to realize a positive ROI within a year. From my experience for any automation project to be a success you need the following: automation experience, test strategy knowledge, and functional understanding of the application under test. Since we are talking testing as a service we can probably assume they have little knowledge of the application under test (unless they wrote it or it's a COTS product) which is your job to provide, without it low value test cases will be delivered and the entire effort and investment will be wasted.
Which tools (by top vendors) can be used to achieve E2B test automation of a Web application?