Regarding submissions, I’ve got a couple major takeaways:
1) Integrate some sort of plugin-check script in order to aid approval process.
2) Differentiate submission vs vetting. Manpower demands are too high to fully vet at submission stage.
This is the section I couldn’t grab a lot of tangible changes from the conversation. Would love some additions to this.
I’m not sure what all really needs changed during the initial submission. It sounds like Otto+crew already beefed up their standards quite a bit recently. Would be great to require some form or “quality control” right out of the gate but at the same time do not want to break the openness of the repo.
I think the vetting submission process could become a community deal though. A simple form with the plugin name/url and a textbox for comments could add plugins to a queue. Those who want to join in on the plugin review team can use that textbox to link to or include their own review of the plugin. As they review a few plugins the current plugin review team can group up and invite them to join?
I think one thing to help the submission process would be a steps + completed score. Like when I joined linked-in, they had things I needed to do, and a % complete. We could have things like:
30% Pass Auto-Security Test
30% Pass Code Quality Test
10% Plugin description
5% Plugin image
15% Screenshots
5% 2 ratings and review
….
whatever the steps might be to having a plugin demonstrate quality and value to users
I like this idea of a initial-completion score a lot. I think if some metrics can be agreed upon, it would immediately encourage plugin developers to do complete documentation around their plugin.
Whatever process goes into place needs to be created carefully so that new developers are not discouraged to quit or even end up moving to another platform. I think getting into the door should still be a fairly simple process, but maybe some help with keeping with standards and major security flaws.
This isn’t ever going to happen due to man power issues but if when submitting your first plugin the reviewer more mentored the developer on how to do things properly. I just don’t think there are many metrics that will actually be agreed on for auto testing to deny a plugin.
Giving nn% rewards is a great idea. Especially if those percentages are used in some way, like impacting where the plugin appears in search results.
@patrickgarman’s point is very important though. We shouldn’t achieve better quality at the cost of discouraging new developers.
Shane Pearlman
8:16 pm on August 16, 2012 Permalink
I hear your concern patrick.
That is why I liked the % idea. Until you hit 100%, you aren’t a real plugin yet (little pinnochio). You’re just a pet project.
You can host your pet project on the repo, but we should lower it presence and value. We could even label it as “experimental” or something. And once you get your act together and hit 100% we’ll give you the bar mitzva and introduce you to the world.
Just to play devils advocate, while we do want to encourage learning and participation, is it reasonable to the user community to have poorly executed, documented and displayed plugins in the repo? I would personally be ok with seeing us go from 15-20 new plugins a day (according to otto) down to 5 if those ones are of better quality and presentation.
Where do you think we should draw the line? Requiring screenshots & a basic writeup in english that explains your plugin is just a baseline for quality in my eyes.
I think a lot of this is getting a little too mixed with more of a ‘vetted’ plugin though. I’m all for wanting better quality but the fact is if a plugin isn’t any good most people will delete it and just move onto another. Too many hurdles will start pushing people away. Even screenshots I would only consider a nice to have. If an author wants their plugins used they will put the effort into making the plugin great, if it isn’t great then it can sit there dormant like so many plugins already do. Don’t get me wrong I want to see improvements in new plugins, but just want to do it very very carefully. Maybe instead of requiring these for submission it could just be a “plugin completion” bar that appears when users look at it, or even just a notice to bug the developer. LinkedIn as the example you still get your profile even though it is only X% complete.
Requiring english I’d disagree with entirely though. Sometimes this just isn’t relevant, Alipay for example is a Chinese payment gateway. Requiring an english description would get in the way of plugins like these… http://wordpress.org/extend/plugins/tags/alipay
It’s an interesting idea, but I think we need to be a little conscious of making the plug-in repo process tedious. Do some of our best developers want to go through a wizard every time they want to share some well constructed code?
I think there could definitely be improvements here in the form of automated testing. As a baseline, an automated test for errors, warnings, notices, etc. could be incorporated. Errors and warnings would cause the plugin to fail, while notices would simply show a notice (similar to the theme checker).
A score could also be generated based on the WordPress coding standards, more as an informative tool for users. I know that many plugin authors don’t seem to use any coding standard, let alone WordPress’ (I myself avoid a few of the parts of the WP coding standards that seem insane), and I think at least giving an indication of this would improve the quality of plugins. A score (as a percentage, e.g.) would give developers a goal to work towards.
Shane Pearlman, and 3 others are discussing. 
Brian 1:16 am on August 16, 2012 Permalink | Log in to Reply
Regarding submissions, I’ve got a couple major takeaways:
1) Integrate some sort of plugin-check script in order to aid approval process.
2) Differentiate submission vs vetting. Manpower demands are too high to fully vet at submission stage.
This is the section I couldn’t grab a lot of tangible changes from the conversation. Would love some additions to this.
Patrick Garman 10:26 am on August 16, 2012 Permalink | Log in to Reply
I’m not sure what all really needs changed during the initial submission. It sounds like Otto+crew already beefed up their standards quite a bit recently. Would be great to require some form or “quality control” right out of the gate but at the same time do not want to break the openness of the repo.
I think the vetting submission process could become a community deal though. A simple form with the plugin name/url and a textbox for comments could add plugins to a queue. Those who want to join in on the plugin review team can use that textbox to link to or include their own review of the plugin. As they review a few plugins the current plugin review team can group up and invite them to join?
Shane Pearlman 1:04 pm on August 16, 2012 Permalink | Log in to Reply
I think one thing to help the submission process would be a steps + completed score. Like when I joined linked-in, they had things I needed to do, and a % complete. We could have things like:
30% Pass Auto-Security Test
30% Pass Code Quality Test
10% Plugin description
5% Plugin image
15% Screenshots
5% 2 ratings and review
….
whatever the steps might be to having a plugin demonstrate quality and value to users
Brian 5:34 pm on August 16, 2012 Permalink
I like this idea of a initial-completion score a lot. I think if some metrics can be agreed upon, it would immediately encourage plugin developers to do complete documentation around their plugin.
Patrick Garman 6:09 pm on August 16, 2012 Permalink
Whatever process goes into place needs to be created carefully so that new developers are not discouraged to quit or even end up moving to another platform. I think getting into the door should still be a fairly simple process, but maybe some help with keeping with standards and major security flaws.
This isn’t ever going to happen due to man power issues but if when submitting your first plugin the reviewer more mentored the developer on how to do things properly. I just don’t think there are many metrics that will actually be agreed on for auto testing to deny a plugin.
Brent 7:42 pm on August 16, 2012 Permalink
Giving nn% rewards is a great idea. Especially if those percentages are used in some way, like impacting where the plugin appears in search results.
@patrickgarman’s point is very important though. We shouldn’t achieve better quality at the cost of discouraging new developers.
Shane Pearlman 8:16 pm on August 16, 2012 Permalink
I hear your concern patrick.
That is why I liked the % idea. Until you hit 100%, you aren’t a real plugin yet (little pinnochio). You’re just a pet project.
You can host your pet project on the repo, but we should lower it presence and value. We could even label it as “experimental” or something. And once you get your act together and hit 100% we’ll give you the bar mitzva and introduce you to the world.
Just to play devils advocate, while we do want to encourage learning and participation, is it reasonable to the user community to have poorly executed, documented and displayed plugins in the repo? I would personally be ok with seeing us go from 15-20 new plugins a day (according to otto) down to 5 if those ones are of better quality and presentation.
Where do you think we should draw the line? Requiring screenshots & a basic writeup in english that explains your plugin is just a baseline for quality in my eyes.
Patrick Garman 3:55 pm on August 17, 2012 Permalink
I think a lot of this is getting a little too mixed with more of a ‘vetted’ plugin though. I’m all for wanting better quality but the fact is if a plugin isn’t any good most people will delete it and just move onto another. Too many hurdles will start pushing people away. Even screenshots I would only consider a nice to have. If an author wants their plugins used they will put the effort into making the plugin great, if it isn’t great then it can sit there dormant like so many plugins already do. Don’t get me wrong I want to see improvements in new plugins, but just want to do it very very carefully. Maybe instead of requiring these for submission it could just be a “plugin completion” bar that appears when users look at it, or even just a notice to bug the developer. LinkedIn as the example you still get your profile even though it is only X% complete.
Requiring english I’d disagree with entirely though. Sometimes this just isn’t relevant, Alipay for example is a Chinese payment gateway. Requiring an english description would get in the way of plugins like these… http://wordpress.org/extend/plugins/tags/alipay
Jake Goldman 4:22 pm on August 22, 2012 Permalink
It’s an interesting idea, but I think we need to be a little conscious of making the plug-in repo process tedious. Do some of our best developers want to go through a wizard every time they want to share some well constructed code?
Ryan McCue 2:53 am on August 16, 2012 Permalink | Log in to Reply
I think there could definitely be improvements here in the form of automated testing. As a baseline, an automated test for errors, warnings, notices, etc. could be incorporated. Errors and warnings would cause the plugin to fail, while notices would simply show a notice (similar to the theme checker).
A score could also be generated based on the WordPress coding standards, more as an informative tool for users. I know that many plugin authors don’t seem to use any coding standard, let alone WordPress’ (I myself avoid a few of the parts of the WP coding standards that seem insane), and I think at least giving an indication of this would improve the quality of plugins. A score (as a percentage, e.g.) would give developers a goal to work towards.