The Marketplace ranking is still broken, and the #1 plugin has 1.1 stars

I have raised this in this forum, in feedback documents, and in direct conversations with Adobe (at least) since 2023. Every time, the answer was some version of “thanks, we’ll review this internally” or “there is currently nobody available to work on this.” Nothing has changed. So here it is again, with today’s screenshots.

Rank 1 under “Most popular” has an average rating of 1.1 out of 5

Pantone Connect. 5,404 ratings. 1.1 stars. Position one. The very first thing every user sees when they open the “All plugins” tab in the Marketplace.

This is what happens when “popular” means nothing but a cumulative install counter, a number that only ever goes up, that no amount of user dissatisfaction can move, and that is self-reinforcing, because whatever sits at the top keeps collecting installs precisely for sitting at the top. Five thousand people took the time to tell Adobe this listing did not work for them, and the ranking system did not register a single one of them.

Think about what that does to a user opening the store for the first time. They install the #1 recommendation, it disappoints them, and the conclusion they draw is not “that one plugin was bad.” It is “the Adobe plugin store is junk.” Every one of us publishing there pays for that impression.

“Top rated” is not a quality ranking either

Sorting by Top rated uses the raw star average and completely ignores how many people voted. One single 5-star rating beats twenty 5-star ratings. It beats three hundred ratings averaging 4.9.

So the “best” plugins on the platform are, by construction, the ones almost nobody has rated. A developer who spent years earning hundreds of genuine reviews ranks below a listing with one vote from a friend (or from himself). The one sorting option that is supposed to reward quality is the one that systematically hides it.

And the cards do not even show the rating count. A user looking at five stars cannot tell whether that means five people or five hundred without clicking into the listing.

This is not a hard problem

That is what makes it so frustrating. This is not a redesign, not a re-architecture, not a roadmap epic. It is two sort keys.

Top rated – a Bayesian weighted average instead of the raw mean:

score = (v / (v + m)) * R + (m / (v + m)) * C

v = number of ratings for this listing
R = average rating for this listing
m = minimum-votes constant (e.g. 20)
C = mean rating across all listings

Listings with few votes get pulled toward the platform average and can no longer win on a single rating. Listings with a real track record keep their own average. IMDb has done it this way for over twenty years. A Wilson lower bound works just as well. Either one is an afternoon of work.

Most popular – factor in the average rating, the number of ratings and the date of the last update, instead of raw installs alone. At the absolute minimum, stop ranking a 1.1-star listing first.

Both views – print the rating count next to the stars.

Who this is addressed to

To be clear: I don’t think the Adobe people reading this forum disagree with me. My impression over the past three years is that you know all of this already, and that the blocker sits above you – nobody is being given the time to touch it.

So consider this post something you can point at. If it helps to have the problem stated bluntly and in public by a developer who has been building on this platform for a decade, please use it.

Adobe built all of this. Why not give it a little love?

Adobe invested a great deal into this platform: the marketplace itself, the review pipeline, the developer distribution portal, the UXP architecture, the pre-release program, the developer relations team, the events. An enormous amount of work went into getting developers to build here.

And then the two sort keys that decide what a user actually sees – the last three metres of that entire pipeline – have been left untouched for years, to the point where the store’s top recommendation is a listing rated 1.1 out of 5. All of that investment is being undercut by something that would take one developer one afternoon.

For whoever has to justify the headcount: this is not a developer-fairness request. The store’s number-one recommendation currently has an average rating of 1.1 out of 5. That is the first impression of the Creative Cloud plugin ecosystem for every new user who opens that tab. Whatever the Marketplace’s conversion and retention numbers look like, the entry point is actively working against them, and the fix is a sort key, not a project.

I understand that teams are stretched and priorities are set elsewhere. But “no resources” has now been the answer for three years running, for a change of this size. At some point that stops being a resourcing issue and starts being a decision.

Is this still being maintained?

Is the Plugins Marketplace still being maintained at all, or has it been effectively abandoned?

I would much rather be told there is hope, and that small fixes like this one can still land.

To the other developers reading this

If you have seen the same thing, please reply, with your listing, your ranking position, your numbers. One person complaining for three years is easy to file away. Fifteen developers in one thread is something that can be taken into a planning meeting. That is what would actually move this.

Previous discussion, same issue, March 2023: How the Plugins Marketplace could be improved – Detailed feedback (section 2.3).

I especially like your point about how the Top Rated sorting section can and should be improved to do a WEIGHTED ranking that factors in the aggregate number of reviews. That would provide a much truer and more accurate assessment of what has actually earned its ranking in those slots.

Most Popular may be a little trickier since that one could just be based on raw number of installs – still if the average rating is abysmal I think there’s a fair point to be made that the plugin clearly is NOT very popular (at least depending on how you define the word.)

I’m curious what the Adobe Exchange Team thinks about your ideas?

There also need to be a small element of time weighting so that new arrivals get some visibility over and above very long standing popular plugins. (Say exponential weighting on a monthly basis … say the new running average is 0.95 of the previous running average plus 0.05 of this month’s score … and perhaps initiallise with 100% of the first month’s score.)

We saw the state of the marketplace and opted to not treat it as a viable distribution channel. The only reason we decided to put our plugin up on marketplace was just so we could technically say “Designed for Premiere Pro” in our marketing materials. This is a high quality post, thank you for making it tomzag.

As far as I know, Marketplace has no product manager right now and there are no plans to do anything about it.

We had a time of development. And we had times of neglect. Now is a period of neglect.

If we want to convince Adobe to improve it, we need to communicate how their customers will benefit from it. And by customers I mean shareholders and people who buy their stock. Not the creatives who buy software.

Also, a significant part of employees’ salaries is paid via stock options. That also plays a role in decisions.

For anyone who is serious about selling plugins: Marketplace is not the place you are looking for, if Adobe does not take it seriously. There are better platforms.

Thank you all.

@TeeJay the time weighting is a good addition and I’d fold it straight into the “Most popular” proposal: an exponentially decayed install/rating score means a listing has to keep earning its position instead of coasting on a counter that only goes up.

@PlanarFlow that is the part I find hardest to read. A publisher listing on the Marketplace only to be allowed to say “Designed for Premiere Pro” is describing a store that has stopped working as a store. I don’t think that’s an unusual position any more.

@Jarda I think you’re right that “this is unfair to developers” is not an argument that gets anything funded, and that whatever we ask for has to be framed in terms of what Adobe gets out of it. Where I’d push back is on the idea that this means talking about shareholders. Plugin revenue is a rounding error at Adobe’s scale.
The version I think does hold up: Adobe wants to be able to say that Creative Cloud is a platform other people build on. That claim is only as good as the storefront that presents it. Right now the storefront’s top recommendation is rated 1.1 out of 5, and publishers like @PlanarFlow are listing there for a marketing badge rather than as a distribution channel. That is an ecosystem story going in the wrong direction, and it is visible to anyone who opens the marketplace, including at the competitors we get compared to.
That’s an argument someone inside can repeat in a meeting without it sounding like a developer complaint.

I like that argument. I think that is a good reframing that managers can weigh in on.

And if anyone proves me wrong, that would be nice as well :smiley: .

Also, Adobe has a new CFO and CEO. Maybe things will be done differently.