Migrating from Drupal to Payload CMS

It's taken a total of two years for us to research and complete our migration away from Drupal. It's been challenging, but also rewarding as the gains are already significant.

What follows is a long-ish post that describes briefly how we made the switch - but more importantly, why we migrated from Drupal  to Payload CMS , as well as the relative strengths and weaknesses of each.

Before we get started it's important to say that there are great people in the Drupal community - great developers, and originally at least, a great framework, but agencies like ours need to respond and adapt to client needs.

The TL;DR of 'how' we migrated — simply enable the JSON:API and Serialization modules in Drupal, and then visit the JSON:API endpoints for your Drupal installation to retrieve collection and node data from which you can write your import scripts. If you need unpublished / draft content, use Puppeteer  to authenticate your requests.

Screenshot of Drupal Modules settings with JSON:API and Serialization enabled.

Visiting an endpoint like /jsonapi/node/article/{uuid} will result in the following... abbreviated here since the complete JSON document is quite long.

{"jsonapi": {"version": "1.0","meta": {"links": {"self": {"href": "http://jsonapi.org/format/1.0/"}}}},"data": {"type": "node--article","id": "dfadb5a3-d680-497f-80ac-4b7476912aa0","links": {"self": {"href": "https://yoursite.com/jsonapi/node/article/dfadb5a3-d680-497f-80ac-4b7476912aa0?resourceVersion=id%3A32"}},"attributes": {"drupal_internal__nid": 32,"drupal_internal__vid": 32,"langcode": "en","revision_timestamp": "2020-01-09T20:26:08+00:00","revision_log": null,"status": true,"title": "Your article title","created": "2020-01-09T20:26:08+00:00","changed": "2024-04-03T03:57:53+00:00","promote": false,"sticky": false,"default_langcode": true,"revision_translation_affected": true,"path": {"alias": "/article-01","pid": 235,"langcode": "en"},"content_translation_source": "und","content_translation_outdated": false,"body": {"value": ..."}}}

At a higher level, and in a bit more detail, here's a rough 'blueprint' for how we'd approach the migration of a Drupal site (of any size), to Payload:

1. Content Mapping: We'd start by auditing and mapping Drupal content types to Payload collections — identify any complex fields or relationships that may require custom fields or hooks in Payload.
2. Field and Relationship Configuration: We'd set up collections and fields in Payload to match the structure of existing Drupal content — defining relationships, localization, and any custom fields as needed.
3. Data Migration: As above — we need to export data from Drupal and Drupal's JSON:API module is a great way to do this, followed by scripts to import this data into Payload.
4. URL and SEO Considerations: We'd need to ensure that URLs remain consistent or set up redirects if necessary. Payload’s flexibility allows us to define custom slugs and SEO configurations to avoid any loss in search engine rankings.
5. User Management: If we need to migrate Drupal users, we'd map them to Payload’s authentication structure. Payload uses a flexible and customizable user collection, which is really just another Payload collection that can be combined with role-based access control and customizable permissions to match Drupal permissions and roles.
6. Rebuild the Front end: Lastly, we'd rebuilt the front end (the webpages including styles and other media assets that the visitors actually see), using a modern framework like Next.js.

Unfortunately, it's not a small task and so Drupal website owners need to weigh the costs and likely benefits before making the switch. What follows is arguably more helpful in this respect.

Here's why as an agency we chose to move away from Drupal. The following is an edited extract from a report we produced for a client several months ago and is of course equally relevant to us as an agency.

1. Why We Migrated From Drupal

Drupal is an open-source content management system (CMS) https://drupal.org , offering a robust platform for developing and managing websites. It emerged in the early 2000s, gaining a reputation for its flexibility and scalability. Drupal's extensive library of modules and themes allows it to cater to a range of website types from complex enterprise-level portals to e-commerce platforms.

Drupal remains an important solution for organizations that require extensive customizations and have the resources to support its development and maintenance. However, in the broader market, Drupal faces stiff competition from other CMS platforms , ‘page building’ platforms like WiX  and Squarespace , as well as from modern 'headless' CMS solutions such as Contentful  and Payload CMS  - solutions that offer greater flexibility in how content is stored, delivered, and presented.

1.1. Drupal Strengths

1. Scalability and Flexibility: Drupal is highly regarded for its ability to scale with the growing needs of a business and for the flexibility its modular system offers.
2. Robust Security: It has a strong security record, making it a preferred choice for government and enterprise websites.
3. Community and Support: Drupal benefits from a large, active community that contributes modules, themes, and distributions.
4. Extensibility: Most of the Drupal framework is 'pluggable', including the localization and language subsystem, which means, for example, that you can create custom localization rules that would allow you to change the default language between areas in a single website. This is something we've done with clients that have multi-country websites and would like the default language of their country areas to change to match the country's principal language.

1.2. Drupal Weaknesses

1. Complexity: Drupal's steep learning curve is one of its most significant barriers to entry. The complexity of its underlying architecture can overwhelm new developers. The entire concept of page rendering, render arrays and field API — is arguably incredibly sophisticated, but again — a very steep learning curve.
2. Development Cost: The complexity and scale of Drupal mean that developing and maintaining a Drupal site often requires more time and expertise than more straightforward systems, leading to higher costs.
3. Outdated in Certain Aspects: Compared to newer, more agile frameworks and headless CMS platforms that utilize modern technologies like JavaScript and React, Drupal can seem outdated. Modern content management solutions offer development experiences that are more in line with current web development practices, potentially making them more attractive to developers looking to enhance their careers with future-proof skills.
4. Long-standing Issues: Certain issues, such as those related to multilingual support in Drupal’s layout builder (https://www.drupal.org/project/drupal/issues/3044386 ), highlight some of the difficulties in dealing with Drupal's large and complex codebase. These issues can take years to address and, in some cases, remain unresolved, posing challenges for development teams and potentially impacting the timely delivery of projects.
5. Drupal is arguably an ‘over engineered’ solution for smaller projects.

1.3. The Thailand Context

In case you haven't noticed, we're a Thailand-based agency. We have local as well as international clients. Whether via us, or from other agencies, hiring agencies in Thailand that offer Drupal development is especially challenging. At the time of writing, there are no officially recognized Drupal Association Supporting Partners with offices in Thailand. Agencies such as Fluxus and Infonomic were previously available as recognized Drupal partners, but no longer offer Drupal services. More importantly, of the single ‘unofficial’ partner listed under Drupal Services at Drupal.org, as well as of the two remaining known agencies that offer some degree of Drupal support — none are what might be considered top-tier, or even second-tier agencies — able to offer ‘one stop’ solutions for Drupal development combined with agency-level identity and graphic design.

1.4. Drupal Summary

In summary, while Drupal remains a powerful and flexible CMS capable of supporting a wide range of web development projects, smaller agencies and teams need to carefully consider the investment required to develop and maintain expertise in Drupal. The platform's complexity, coupled with the evolving landscape of web development technologies, suggests that exploring modern alternatives might be more beneficial for those looking to stay at the forefront of the industry. Further, within the Thailand context, the procurement of Drupal services is made particularly difficult by the lack of agency-level Drupal support.

2. Why We Chose Payload CMS

We were looking for a 'right-size' CMS that would allow us to build quickly, and adapt to rapidly changing user requirements.

Payload CMS  is an open-source, headless content management system designed with developers in mind. It offers a range of features that cater to both development flexibility and user experience.

2.1. Payload Strengths

1. Configuration: Payload's configuration system is incredibly powerful, very easy to customize, and most importantly, easy to reuse. We can take our 'base' configuration and simply copy it into a new project and have the 'admin' backend for a new client project up and running in minutes.
2. Localization: Payload CMS's field-level localization strategy was exactly right for the multilingual projects we regularly work on. This was one of the main factors in our decision to use Payload CMS. There were real difficulties with localization in other systems we reviewed, and in some cases real costs associated with offering more than one locale.
3. Extensible: We could see the potential to integrate Payload with our text and search management features, in particular Solr  — as this is an important part of our collections management offerings for research and project-based clients - an example of which you can see here in the FORRU Library .
4. Editor Choice: Although initially based on the Slate rich text editor, it was clear that Payload's flexibility meant that it would be possible to implement a custom editor. We've previously built custom text serializers and editors with CKEditor, Slate and others, and were very interested in Lexical  — the open-source editor framework from Meta . Again, the importance of our text indexing and search features meant that a solid rich text editor strategy was important to us as an agency. While researching Lexical we discovered Alessio Gravili's  work on a Lexical editor plugin for Payload — which was exactly what we were looking for. Alessio has since joined the team at Payload, and thanks to his efforts, Lexical is now part of core at Payload and the default editor going forward.
5. Field API and Admin UI: While still evolving, the field and schema configuration for collections and corresponding admin UI — is superb. The UI kit in particular is one of the best out there.
6. Uploads: While not yet a replacement for a complete media management solution, being able to convert 'regular' collections into 'upload' collections is very nice. Corresponding image size generation and built-in image editing is excellent.
7. Our Data: Maybe we're old fashioned, but we're glad that Payload lets us own, store, retrieve and even convert our client data - in particular with Lexical since Lexical stores editor content in a structured format (and not simply chunks of HTML).
8. Next.js: Payload CMS is now tightly integrated with Next.js which means that should we wish to — we can deploy the admin dashboard, local API and web front end in a single Next.js application.
9. Open Source: The codebase for Payload CMS is open source, which is great when considering deeper integration, or simply trying to understand how things work. Contributions are also a great way to help shape the future of Payload.

2.2. Payload Weaknesses

1. Complexity in Presentation: As they say, with great power comes great responsibility. Payload CMS's developer friendly approach also means that developers must take greater responsibility for how they present data. This means building a serialization strategy and component collection that will take Payload CMS blocks and fields, and convert them into something meaningful on the web (or wherever your client happens to be). It's very powerful, and the team at Payload have taken the unprecedented step of open sourcing their own website and CMS  , which combined with other starter templates and libraries means there are plenty of 'sign posts' on how to get started, but, as an agency you'll need to understand and successfully implement your own font-end component serialization strategy.
2. Media Management: As mentioned above, one of the interesting things about Payload is the way in which you can convert a 'regular' document collection into an 'upload' collection. This works great, however, it's not yet a complete media management solution. We attempted to write a custom collection view for media / photos but are waiting for the 3.0 release to stabilize before looking at this again. I suspect there will also be more to come from the Payload team in this area soon.
3. Next.js: A strength above, but a weakness if you're not a Next.js shop — in which case you'll likely need to deploy the Payload CMS admin dashboard separately from your client front-end application, and while this might be a wise strategy from a security perspective, it's still something to consider.
4. Open Source: While a strength above, it's also worth considering that Payload CMS is an MIT  licensed project. Payload is also a VC funded project, which means that the team are working hard for a return on their investment, as well as a return for their investors. This translates into a project that is moving and innovating quickly, but is also different philosophically from a GPL  licensed project like Drupal, or an open source project that is managed via a foundation with a corresponding governance structure. It also means there is a delicate balance required - between encouraging community contributions, and the risk that such encouragement is perceived as an attempt to solicit free labor. Overall we feel it's a net win - but still, something to keep in mind.

2.3. Payload Summary

In summary, Payload CMS has emerged as the ideal content management solution for our agency's needs. Its developer-centric design empowers us to build and adapt projects rapidly, aligning perfectly with our goal to respond swiftly to client demands. The powerful and reusable configuration system allows us to set up an admin backend for new projects in minutes, significantly reducing our development time.

Field-level localization addresses one of our primary challenges — managing multilingual content. The extensibility of Payload CMS has also been a significant advantage. It seamlessly integrates with our text and search management tools, enhancing our collections management offerings for research and project-based clients.

Integration with Next.js streamlines our deployment process by allowing the admin dashboard, local API and web frontend to coexist within a single application.

While Payload CMS requires developers to take greater responsibility for front-end data presentation — necessitating a solid serialization strategy — we view this as an opportunity to create more tailored and optimized user experiences.

The open-source nature of the project provides us with transparency and the flexibility to perform deeper integrations when necessary. However, we remain cognizant of the project's MIT licensing and VC-funded status, understanding that it differs philosophically from GPL-licensed projects like Drupal.

Ultimately, Payload CMS offers a balanced blend of flexibility, modern tooling, and efficiency and has enabled us to overcome the limitations we faced with Drupal.

3. Conclusion

Our journey from Drupal to Payload CMS has been both challenging and rewarding. The decision to migrate was not taken lightly, but it became clear that staying with Drupal was hindering our ability to adapt and grow in response to client needs. Drupal's complexity and the scarcity of qualified Drupal developers made it increasingly impractical for us to use.

Payload CMS has revitalized our development process. Its modern, developer-friendly approach aligns with current web development practices, allowing us to build quickly without sacrificing quality or scalability. The immediate gains in efficiency and flexibility have already had a positive impact on our projects and client satisfaction.

Looking forward, we are excited about the possibilities that Payload CMS opens up for us and our clients — not least of which will include the integration of 'first-pass' automatic content translation via generative AI.