Quantcast
Channel: Yet Another Tridion Blog
Viewing all 215 articles
Browse latest View live

Enable XPM Session Preview in Web 8 using Microservices

$
0
0
If you implemented the steps in the previous post Implementing Experience Manager in DD4T 2.0 .NET, then at this moment you can run your web application and XPM markup will appear in your HTML output. Also XPM functionality is working.

However, if there are modifications on the Page or any Components on the Page that are not published, the "Update Preview" button will flash in the XPM GUI. Pressing this button will eventually end up in an error and the Update Preview button will keep flashing. This is due to the fact that we have not configured Session Preview yet.

The following sections will guide you through installing and configuring Session Preview and database and services it uses.

Experience Manager Database

This database, aka Session Preview Database, has the same structure as a Tridion Content Data Store database, only that it is used as temporary storage for items that have been modified in XPM and previewed (i.e. rendered with templates), but not yet published.

Consult SDL Documentation portal link Creating the Experience Manager database about steps for creating this database.

Discovery Service

SDL Web 8 introduced several Content Delivery microservices, one of them being the Discovery Service. This service is in charge with showcasing the capabilities a particular server has, such that it is known to other client APIs, which service is installed, which version and under which URL it is accessible.

In my implementation, the Discovery Service runs on port 8082, under URL path: /discovery.svc

For an XPM implementation, a default Discovery Service is used. The service provides an endpoint that issues authorization tokens for clients that provide certain user/password combinations. These tokens are to be used when communicating with the other (discoverable) services. This endpoint is accessible under URL path: /token.svc

Consult SDL Documentation portal link Installing the Discovery Service about steps for installing this service.

Once configured and the discoverable services registered, the Discovery Registration Tool lists the following services in my environment:

[ {
"Capability" : "com.sdl.web.discovery.datalayer.model.ContentServiceCapability",
"id" : "DefaultContentService",
"lastUpdateTime" : 16663565840,
"uri" : "http://services.tridion.com:8081/content.svc"
}, {
"Capability" : "com.sdl.web.discovery.datalayer.model.TokenServiceCapability",
"id" : "DefaultTokenService",
"lastUpdateTime" : 16663565996,
"uri" : "http://services.tridion.com:8082/token.svc"
}, {
"Capability" : "com.sdl.web.discovery.datalayer.model.PreviewWebServiceCapability",
"id" : "DefaultPreviewWebService",
"lastUpdateTime" : 16663565934,
"uri" : "http://services.tridion.com:8083/ws/preview.svc"
} ]

Session Content Service

The Content Service is in charge with serving content and other functionality for your delivery API. In order to use XPM with this service, a different flavour should be installed, namely the Session-enabled Content Service or for short Session Content Service.

This service is not directly used by XPM itself, rather it is used by the web application and the Tridion Content Delivery API when it reads content, metadata, links, etc. from the Content Data Store. In my implementation this service runs on port 8081 under URL path /content.svc

DD4T uses the Content Service by means of pointing to the Discovery Service this service is registered with:

<appSettings>
<addkey="discovery-service-uri"
value="http://services.tridion.com:8082/discovery.svc/"/>
...

Consult SDL Documentation portal link Installing the Session-enabled Content Service about steps for installing the Session Content Service.

By default this service comes with OAuth enabled, which makes it harder to debug or to at least check if it's working. Go ahead and check below the section Disable OAuth for Easier Debugging, in order to disable OAuth for debugging purposes.

Configure Experience Manager Database

Session Content Service must know of the Experience Manager Database in order to read the content from the published data store or the XPM data store. This means we need to configure it in its cd_storage_config.xml. Place for following node and configure its relevant values inside node Configuration / Global / Storages / Wrappers:
<WrapperName="SessionWrapper">
<StorageType="persistence"Id="sessionDb"dialect="MSSQL"Class="com.tridion.storage.persistence.JPADAOFactory">
<PoolType="jdbc"Size="10"MonitorInterval="60"IdleTimeout="120"CheckoutTimeout="120"/>
<DataSourceClass="com.microsoft.sqlserver.jdbc.SQLServerDataSource">
<PropertyName="serverName"Value="db.tridion.com"/>
<PropertyName="portNumber"Value="1433"/>
<PropertyName="databaseName"Value="xpm-db"/>
<PropertyName="user"Value="XPMUser"/>
<PropertyName="password"Value="password"/>
</DataSource>
</Storage>
</Wrapper>

Session Preview Service

This microservice is a key component of the XPM architecture and without it, XPM will not work. The Preview Service is in charge with saving changes to content and metadata back to the Tridion Content Manager. In my implementation this service runs on port 8083 and it is accessible under URL path: /ws/preview.svc

Consult SDL Documentation portal link Installing the Preview Service about steps for installing the Session Preview Service.

By default this service comes with OAuth enabled, which makes it harder to debug or to at least check if it's working. Go ahead and check below the section Disable OAuth for Easier Debugging, in order to disable OAuth for debugging purposes.

Configure Experience Manager Database

Just like its Content Service counterpart, the Preview Service must know of the Experience Manager Database in order to read the content from the published data store or the XPM data store. This means we need to configure it in its cd_storage_config.xml. Place for following node and configure its relevant values inside node Configuration / Global / Storages / Wrappers:

<WrapperName="SessionWrapper">
<StorageType="persistence"Id="sessionDb"dialect="MSSQL"Class="com.tridion.storage.persistence.JPADAOFactory">
<PoolType="jdbc"Size="10"MonitorInterval="60"IdleTimeout="120"CheckoutTimeout="120"/>
<DataSourceClass="com.microsoft.sqlserver.jdbc.SQLServerDataSource">
<PropertyName="serverName"Value="db.tridion.com"/>
<PropertyName="portNumber"Value="1433"/>
<PropertyName="databaseName"Value="xpm-db"/>
<PropertyName="user"Value="XPMUser"/>
<PropertyName="password"Value="password"/>
</DataSource>
</Storage>
</Wrapper>

Publication Target

For XPM to work, it must be enabled first in the Publication Target settings. Open your Staging target and select the checkbox Enable for inline editing:
Publication Target enabled for Experience Manager
Once this checkbox is selected, a new tab appears Session Preview:
Session Preview settings

Each line has the following meaning:
  • Content Delivery Endpoint URL: fully qualified URL of the Session Preview Service. E.g. http://my-url:8083/ws/preview.svc
  • OData Access Token URL: in case the services use authentication, this is the fully qualified URL of the Discovery Service token endpoint that performs the check and issues access tokens. E.g. http://my-url:8082/token.svc
  • User Name: user that requests access for the Session Preview Service. This user is defined in file cd_ambient_conf.xml on the Preview Service and must have user role cm
  • Password: password for the user
  • Website URL's: at least one URL that identifies the beginning of an XPM page URL for this staging target

Oh, and Don't Forget...

CIL (REST) Service-Oriented Implementations

Both Content Service and Preview Service require the presence of a cookie named preview-session-token. This cookie identifies the current XPM session and is essential in merging published content from the Content Data Store with not-yet-published content from the Experience Manager Database.

The absence of this cookie has as effect the incorrect functioning of "Update Preview" button in XPM. More precisely, XPM will identify there are modifications to the Page and/or Component on a page that have not been published yet. This triggers the flashing of "Update Preview" button, indicating some new content is available that has not been published yet. Without a valid preview-session-token cookie, the Session Content Service is unable to pull these not-yet-published modifications from Experience Manager Database or from the Content Manager and therefore, Update Preview functionality fails.

The solution is quite straight-forward: install Ambient Data Framework in the web application (where the CIL API is running) so that ADF makes this cookie available in the context when a REST request is built and sent to the Session Content Service.

Alternatively, if using a custom REST client to talk to the Content Service, include the preview-session-token cookie in your custom request.

For more information on architecting XPM solution in service oriented environment, refer to my post Experience Manager in DD4T with Service-Oriented Architecture Providers.

Disable OAuth for Easier Debugging

By default, all microservices in Web 8 have OAuth token based authentication/authorization enabled. This makes it harder to debug or at least inspect by means of simply accessing their endpoint URL in a browser. Attempting to do so, will result in the following JSON being displayed in the browser:

{"error":"invalid_grant"}

Although this message alone is a good indication there is a service listening at that address, port and URL path, it is still very cryptic and does not provide a clear picture what the status of that service is.

This is why, for the duration of configuring, setting up the services, I choose to disable OAuth on the Web 8 microservices. The moment when I have my services up and running, I immediately turn OAuth back on. Note that all services must have the same state of OAuth. You cannot have some services use OAuth and some not. It is either on for all services in an environment or off for all.

There are other methods of debugging / inspecting these services and I won't go into details on those. This is what I prefer doing for a short period of time while getting all functionality running.

To disable OAuth, edit each service cd_ambient_conf.xml file. Locate node RequestValidator and comment it out:

<RequestValidator>com.sdl.web.oauth.validator.OAuth2RequestValidator</RequestValidator>

<!--RequestValidator>com.sdl.web.oauth.validator.OAuth2RequestValidator</RequestValidator-->

Next, locate node Rules and set its Enabled attribute to false:

<RulesEnabled="false"/>

Restart each service.

To enable OAuth, remove comment from RequestValidator node and set Rules to Enabled true.




Troubleshoot a DXA Java Module

$
0
0
I was asked recently to troubleshoot a DXA 1.5 module that was not behaving properly. The apparent issue was that nested embedded fields in all models were not filled in with values once the Semantic Mapper would execute and create the models.

This prompted me to do a little bit of digging into the Semantic Mapper and its registries of model entities, property and field maps. It turned out the models in the custom module were not being scanned and as such not registered as entities by the Semantic Mapper.

The Semantic Mapper is DXA's de facto model factory that converts a Generic Page or Component into a specific page or component model, by mapping fields in the generic object to properties in the specific object.

The solution was to simply call method registerEntities on the SemanticMapperRegistry singleton and pass it the base package name. The logic will then scan for entity models (i.e. classes that extend DXA's AbstractEntityModel class) in sub-packages and register them for later use by using reflection to extract Entity maps, Property maps and Field maps.

@PostConstruct
privatevoidregisterModels(){
semanticMappingRegistry.registerEntities(getClass().getPackage().getName());
}

I placed the code-above in the model initializer class, which I present below. The semanticMappingRegistry is an autowired singleton that Spring framework injects into the initializer class upon instantiation.

package com.mihaiconsulting.dxa.emerald;

importcom.mihaiconsulting.dxa.emerald.model.entity.MainNavigation;
importcom.sdl.webapp.common.api.mapping.semantic.SemanticMappingRegistry;
importcom.sdl.webapp.common.api.mapping.views.AbstractInitializer;
importcom.sdl.webapp.common.api.mapping.views.ModuleInfo;
importcom.sdl.webapp.common.api.mapping.views.RegisteredViewModel;
importcom.sdl.webapp.common.api.mapping.views.RegisteredViewModels;
importorg.springframework.beans.factory.annotation.Autowired;
importorg.springframework.stereotype.Component;

importjavax.annotation.PostConstruct;

@org.springframework.context.annotation.Configuration
publicclassEmeraldInitializer{

@RegisteredViewModels({
@RegisteredViewModel(viewName ="MainNavigation", modelClass = MainNavigation.class)
})
@Component
@ModuleInfo(name ="Emerald module", areaName ="Emerald")
publicstaticclassEmeraldViewInitializerextends AbstractInitializer {

@Autowired
private SemanticMappingRegistry semanticMappingRegistry;

@Override
protected String getAreaName(){
return"Emerald";
}

@PostConstruct
privatevoidregisterModels(){
semanticMappingRegistry.registerEntities(getClass().getPackage().getName());
}
}
}

A more in-depth explanation and tutorial on how to create a DXA 1.5 Java module is available in my next post Creating a DXA Java Module.



Creating a DXA Java Module

$
0
0
In my previous blog post Troubleshoot a DXA Java Module, I presented a solution for a very specific issue with loading a DXA Java module. Then I had the idea for this blog post -- write a short tutorial on how to write a DXA module.

What follows are the steps to create a new DXA Java module (I am using DXA v1.5). In this post, I present the Content Manager items needed for the new module. In fact, these are the minimum items and properties needed for having a new working DXA module.

In next post Creating a DXA Java Module (part 2), I present the Java code and configuration needed to run the new DXA module in a web-application.

1. Create a New Module in Tridion Content Manager

I am using DXA reference implementation, and I created a new module called Emerald, next to the Core module of the reference implementation. Create the following Tridion folder structure starting from Folder Emerald:


2. Create Module Configuration Component

In Folder Admin, create new Component Emerald on Schema Module Configuration. For minimum configuration, it's enough to only fill in the the mandatory fields.



3. Publish Module Configuration

If using the DXA reference implementation, publish page Publish Settings from Structure Group _System. You don't need to do anything else.

Otherwise, create a Page on Page Template JSON, File Name settings.


Add one Component Presentation to this Page. The Module Configuration component Emerald. If you have more than one module, only place one module Component on this Page, otherwise you will get errors during publishing.


Publish the Publish Settings page.

4. Create Schemas

In Folder Editor, I'm only going to create a Main Navigation Schema that points to an embeddable Schema Navigation Links.


It is very important to give it a unique Root Element Name. I chose MainNavigation and NavigationLinks respectively.


In order to keep things simple, I chose the already predefined target namespace http://www.sdl.com/web/schemas/core. This name is used by the Semantic Mapper vocabularies and it coincides with the Core module vocabulary name. If not define, you must define it in a vocabulary Application Data -- not a nice thing to do. Luckily, we can re-use the existing predefined namespace for the Core module.


5. Create Content

In Folder Editor, create a simple Component Main Navigation on Schema Main Navigation.



6. Create a Component Template

Create a Component Template Main Navigation CT and link it to the Schema Main Navigation. Make it use Metadata Schema Component Template Metadata and fill in the view to use Emerald:MainNavigation. According to DXA naming convention, the view name is prefixed with the name of the module, separated by colon (:).


7. Create a Content Page

Create a Page using File Name Navigation Test Page and Page Template Section Page. Add one Component Presentation to it using Component Main Navigation and Component Template Main Navigation CT.



Publish the Page.

At this moment, the Content Delivery Datastore has all the information necessary to support the DXA web application in fetching the model and rendering the Page. But more about that, in the next post.



Creating a DXA Java Module (part 2)

$
0
0
In my previous post Creating a DXA Java Module, I started presenting the steps for creating the Tridion items on the Content Manager needed for the new DXA module. In this current post, I present the Java code and configuration for having the DXA module run in a web-application.

I wrote my code as a separate JAR using IntelliJ and then running the DXA v1.5 web-application on Tomcat as part of the standard DXA reference implementation project dxa-web-application-java (available in GIT at https://github.com/sdl/dxa-web-application-java/tree/release/1.5).

What follows are the steps on how to create the DXA Java module classes and configurations.

1. Create the IntelliJ Emerald Module

In IntelliJ, create a new Java Module and enable Maven for it. Add the following dependencies in its pom.xml file:

<dependencies>
<dependency>
<groupId>com.sdl.dxa</groupId>
<artifactId>dxa-common-api</artifactId>
</dependency>

<!-- Servlet API -->
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
</dependency>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>jstl</artifactId>
</dependency>

<!-- Spring Framework -->
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-core</artifactId>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-webmvc</artifactId>
</dependency>
</dependencies>

The final layout of the DXA module looks like the following:


2. Module Initializer Class

Configuration class EmeraldInitializer defines the mapping between ViewModels and class or domain models.

It extends AbstractInitializer, so it defines the getAreaName method, which returns the name of this module (i.e. Emerald).

Last, but not least it triggers the registration of the module's entities with the SemanticRegistryMapper, by defining method registerModels as a Spring post construct initialization method. Without this feature, only the entity models defined in the view-domain mappings are going to be registered with the Semantic Mapper. This has as undesired effect, the loss of embedded entities being added to the Semantic Mapper registries. As such, there will be no values for the embedded fields once the model is built.

package com.mihaiconsulting.dxa.emerald;

importcom.mihaiconsulting.dxa.emerald.model.entity.MainNavigation;
importcom.sdl.webapp.common.api.mapping.semantic.SemanticMappingRegistry;
importcom.sdl.webapp.common.api.mapping.views.AbstractInitializer;
importcom.sdl.webapp.common.api.mapping.views.ModuleInfo;
importcom.sdl.webapp.common.api.mapping.views.RegisteredViewModel;
importcom.sdl.webapp.common.api.mapping.views.RegisteredViewModels;
importorg.springframework.beans.factory.annotation.Autowired;
importorg.springframework.stereotype.Component;

importjavax.annotation.PostConstruct;

@org.springframework.context.annotation.Configuration
publicclassEmeraldInitializer{

@RegisteredViewModels({
@RegisteredViewModel(viewName ="MainNavigation", modelClass = MainNavigation.class)
})
@Component
@ModuleInfo(name ="Emerald module", areaName ="Emerald")
publicstaticclassEmeraldViewInitializerextends AbstractInitializer {

@Autowired
private SemanticMappingRegistry semanticMappingRegistry;

@Override
protected String getAreaName(){
return"Emerald";
}

@PostConstruct
privatevoidregisterModels(){
semanticMappingRegistry.registerEntities(getClass().getPackage().getName());
}
}
}

Notice the mapping between view MainNavigation and model class MainNavigation.java. I will present those classes below.

3. The Models

Emerald module defines only two simple model classes MainNavigation and NavigationLinks. These are simple Java beans, which need to contain annotations that help the Semantic Mapper fill their properties with values from generic DD4T models.

The MainNavigation class is annotated with entityName MainNavigation which is in fact the Schema Root Element Name that maps to this model class. It also defines the vocabulary to use while mapping; this is the Schema target namespace defined in Tridion Content Manager, and the standard vocabulary prefix tri.

The MainNavigation.class is also used in the mapping in EmeraldInitializer between the view model and the class model that serves it.

Next, the class extends AbstractEntityModel class which helps identifying it as a model entity and provides additional metadata for XPM and other ids.

Finally, each property is annotated with mapping information from the Schema field using the vocabulary prefix and field XML name.

package com.mihaiconsulting.dxa.emerald.model.entity;

importcom.sdl.webapp.common.api.mapping.semantic.annotations.SemanticEntity;
importcom.sdl.webapp.common.api.mapping.semantic.annotations.SemanticProperty;
importcom.sdl.webapp.common.api.model.entity.AbstractEntityModel;

importjava.util.List;

@SemanticEntity(entityName ="MainNavigation", vocabulary ="http://www.sdl.com/web/schemas/core", prefix ="tri")
publicclassMainNavigationextends AbstractEntityModel {

@SemanticProperty("tri:name")
private String name;

@SemanticProperty("tri:links")
private List<NavigationLinks> links;

public String getName(){
return name;
}

publicvoidsetName(String name){
this.name= name;
}

public List<NavigationLinks>getLinks(){
return links;
}

publicvoidsetLinks(List<NavigationLinks> links){
this.links= links;
}
}

The embedded entity NavigationLinks is presented below. Notice that it is an entity on its own that also extends AbstractEntityModel.

package com.mihaiconsulting.dxa.emerald.model.entity;

importcom.sdl.webapp.common.api.mapping.semantic.annotations.SemanticEntity;
importcom.sdl.webapp.common.api.mapping.semantic.annotations.SemanticProperty;
importcom.sdl.webapp.common.api.model.entity.AbstractEntityModel;

@SemanticEntity(entityName ="NavigationLinks", vocabulary ="http://www.sdl.com/web/schemas/core", prefix ="tri")
publicclassNavigationLinksextends AbstractEntityModel {

@SemanticProperty("tri:url")
private String url;

public String getUrl(){
return url;
}

publicvoidsetUrl(String url){
this.url= url;
}
}

4. JSP View Model

The ViewModel is called MainNavigation as per the mapping present in EmeralInitializer class when registering the view with the domain model. The view name is also referenced from the Component Template metadata defined in Tridion. The Metadata Schema field view defines the DXA module name and JSP view name in the format ModuleName:ViewName.

DXA uses a naming convention referring to the location of JSP views. They must be under path WEB-INF / Views / DXA-Module-Name / Entity. This makes the path of our view /WEB-INF/Views/Emerald/Entity.

Also, since this IntelliJ Java module is packaged as a JAR, the JSP views must be available in the JAR's classpath as resources. As such, we must put the JSP in the JAR under path /META-INF/WEB-INF/Views/Emerald/Entity/MainNavigation.jsp.

<%@ page language="java" contentType="text/html; charset=UTF-8" pageEncoding="UTF-8"%>
<%@ taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core"%>

<jsp:useBean id="entity" type="com.mihaiconsulting.dxa.emerald.model.entity.MainNavigation" scope="request"/>

<div>
<p>Name: ${entity.name}</p>

<c:iftest="${not empty entity.links}">
Links:
<ul>
<c:forEachvar="link"items="${entity.links}">
<li>${link.url}</li>
</c:forEach>
</ul>
</c:if>
</div>

The JSP/JSTL code in the view is trivial. It simply outputs the fields of the MainNavigation model and its multi-valued links. Notice how the entity bean is read from the request.

5. Spring Configuration

One additional configuration is required that indicates Spring framework should process the new DXA module for Spring annotations. We must tell Spring to scan for components under a certain package. In this case, we tell Spring to scan package com.mihaiconsulting.dxa.emerald and process the beans, controllers, auto-wired properties, post construct initialization methods.

DXA imports all /META-INF/spring-context.xml files it finds in all classpaths. As such, I defined one such file and placed it in the JAR's /META-INF/ folder.

<?xml version="1.0" encoding="UTF-8"?>
<beansxmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:context="http://www.springframework.org/schema/context"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/context
http://www.springframework.org/schema/context/spring-context.xsd">

<context:component-scanbase-package="com.mihaiconsulting.dxa.emerald"/>

</beans>

6. Run the Web-Application

Having all these configurations and classes, we can now run the web-application. Everything should be in place now to retrieve and build the models, then to render them in the JSP view:




Toolkit - A Tridion File System Broker

$
0
0
In the following series of posts, I am describing a custom file system storage, called the Toolkit, that mimics the behaviour of the Tridion File System Broker. There are a couple of noticeable differences from the File System Broker in that it:
  • stores items as JSON files on the file system;
  • provides dynamic queries functionality;

The Toolkit offers the following functionality:
  • Dynamic linking (Component, Page and Binary)
  • Component Presentation Factory
  • Component Presentation Assembler
  • Custom Tags execution during Component Presentation assembling
  • Dynamic Content Queries based on item's CustomMeta

Backing the Toolkit implementation are the following:
  • Model Factory offering CRUD operations on models backed by a cache for model objects and a file system provider acting as data layer abstraction;
  • File System Provider acting as Data Abstraction Layer that offers CRUD operations between model objects, JSON de-/serializer, JSON files, and the underlying file system;
  • CacheFactory offering EHCache memory storage for frequently used models;
  • Cache invalidation mechanism that removes stale items from cache;
  • Custom indexes and handling logic that provides performance for content queries;
  • Tridion Deployer module and Tridion Storage extension that publish/unpublish the JSON files from the file system;

Each of the modules above are described in following posts.

The Toolkit is written in plain Java without using any additional framework or third party data access / indexing API. The Toolkit does use FasterXML's Jackson JSON API in its interaction with JSON.

Toolkit is available as open source on the GitHub project of this blog (https://github.com/mitza13/yet-another-tridion-blog/tree/master/File System Toolkit). The distributable JARs are available in its /distributable folder.

Some performance metrics of the Toolkit can be found in post Toolkit - Performance.

Installation and configuration steps can be found in post Toolkit - Installation and Configuration.

Examples for using the Toolkit API can be found in post Toolkit - Examples.



Toolkit - Performance

$
0
0
This post if part of a series about the File System Toolkit - a custom content delivery API for SDL Tridion.

This post presents performance data that was captured for each major functionality with and without caching Linking, CP Assembler, Component Presentation Factory, Dynamic Content Queries, Model Factory.

The data was captured on a 2014 Macbook Pro 15", 16 GB RAM, 2.6 GHz Intel Core i7 running OS X El Capitan.

Test methodology: each test was run for 3 minutes and the total number of successful Toolkit API calls was measured. Then the number of calls per second was computed 'with cache' and 'without cache' test runs. Then a cache boost factor was calculated by diving (the number of API calls with cache) / (number of API calls without cache).

Each cache test was executed 3 times, with different cache time-to-live values of 1 second, 5 seconds and 0 seconds (eternal cache, no expiration). The rationale is to see what impact different cache expiration/eviction values have on the overall performance.

Then finally the entire suite of tests was run twice in order to calculate averages.

Model Factory

Only read model operations were calculated. The reasoning is that Deployer extension is the only one that creates / writes the JSON files and as such that operation is much less frequent than read operations that are used all the time during the normal functioning of the Toolkit.

Models read per second: 1.9m (with cache) and 26k (without cache)
Cache boost: 73x

Model Factory read performance


Link Factory

The test executed component, page and binary links. Each LinkFactory method was executed and counts as one API call.

Links resolved per second: 383k (with cache) and 26k (without cache)
Cache boost: 15x

Link Factory resolve performance


Component Presentation Factory

The test executed getComponentPresentation and getComponentPresentationWithHighestPriority factory calls. Each call counts as one API call. Each factory calls makes one ModelFactory read call, so the times below include also the ModelFactory call duration.

CPFactory calls per second: 3.2m (with cache) and 24k (without cache)
Cache boost: 135x

Component Presentation Factory read performance

Component Presentation Assembler

The test executes a getComponentPresentation factory method which counts as one API call. The call includes a CPFactory call, so the times below include also the CPFactory call duration.

CPAssembler calls per second: 55k (with cache) and 17k (without cache)
Cache boost: 3x

Component Presentation Assembler performance

Dynamic Content Query

The dynamic content queries are not cached. Therefore, the times below are pretty much the same. The tests were executed using different combinations of query criteria (custom meta, schema, date ranges, pagination, and vs or complex queries). Each query counted as one API call. The numbers below represent an average duration of each of these queries. No models were created during the dynamic queries, the queries simply returned the TcmUri of the identified items.

Dynamic queries per second: 6.8k

Dynamic Query performance


Toolkit - Installation and Configuration

$
0
0
This post if part of a series about the File System Toolkit - a custom content delivery API for SDL Tridion.

The File System Toolkit consists of two parts: a Deployer extension and the Content Delivery API itself. This post describes installing and configuring both.

Toolkit JARs are available under the GitHub repository of the File System Toolkit project, in folder /distributable. Alternatively, you can build your JARs using the sources available on GitHub.

Toolkit Deployer Installation

This deployer extension handles the creation of JSON files on the file system. The Toolkit CD API then reads these JSON files and creates model objects from them.

In order to obtain these JSON files, a deployer extension must be installed on top of a functional Tridion Content Delivery Deployer. The Toolkit deployer extension works for both file system-based or database-based Tridion Deployers.

1. In the Tridion Deployer /lib folder, copy files:
  • toolkit-api.jar
  • toolkit-deployer.jar
  • toolkit-model.jar
2. Also to folder /lib, copy files:
  • ehcache-2.8.3.jar
  • jackson-annotations-2.6.4.jar
  • jackson-core-2.6.4.jar
  • jackson-databind-2.6.4.jar
  • logback-classic.jar
  • logback-core.jar
  • slfj-api.jar
3. Edit or create if it doesn't exist, file toolkit.properties and place it in the class-path of the Deployer (/WEB-INF/classes for an HTTP deployer). The following properties are relevant for deployer extension:
  • cacheEnabled - boolean whether to use cache or not;
  • cacheMaxEntriesLocalHeap - integer the size of the cache;
  • cacheMonitorSeconds - integer the interval in seconds to perform cache stale checks;
  • cacheTimeToLiveSeconds - integer the number of seconds to keep an item in cache;
  • cacheTimeToIdleSeconds - integer the number of seconds to keep an idle item in cache;
  • cleanup - boolean whether to delete empty directories after unpublish;
  • contentRoot - string the path where the published Tridion pages are published (this path is defined in cd_storage_conf.xml);
  • jsonRoot - string the path where the JSON files are stored under;
  • prettyPrintJson - boolean whether to space indent JSON files for better human-reading or not;
4. Edit file cd_deployer_conf.xml. Add the following lines before the end tag </Processors>

<!-- Toolkit JSon Modules -->
<ProcessorAction="Deploy"Phase="post-transaction"Class="com.tridion.deployer.Processor">
<ModuleType="PageDeploy"Class="com.mitza.toolkit.deployer.JSonPageDeploy"/>
<ModuleType="ComponentDeploy"Class="com.mitza.toolkit.deployer.JSonComponentDeploy"/>
<ModuleType="ComponentPresentationDeploy"
Class="com.mitza.toolkit.deployer.JSonComponentPresentationDeploy"/>
<ModuleType="BinaryDeploy"Class="com.mitza.toolkit.deployer.JSonBinaryDeploy"/>
</Processor>

<ProcessorAction="Undeploy"Phase="post-transaction"Class="com.tridion.deployer.Processor">
<ModuleType="PageUndeploy"Class="com.mitza.toolkit.deployer.JSonPageUndeploy"/>
<ModuleType="ComponentPresentationUndeploy"
Class="com.mitza.toolkit.deployer.JSonComponentPresentationUndeploy"/>
</Processor>
<!-- End of Toolkit JSon Modules -->

5. Edit file cd_storage_conf.xml. Add the following lines after the opening tag <Storages>

<StorageBindings>
<Bundlesrc="toolkit_dao_bundle.xml"/>
</StorageBindings>

6. Restart the Deployer

Toolkit API Installation

The Toolkit is an API to be used in your web-application as middle-tier to retrieve content published from SDL Tridion. The installation implies copying the JARs and configuration files to folders available in the class-path of your web-application:

1. In your web-application /lib folder, copy files:
  • toolkit-api.jar
  • toolkit-dynamic.jar
  • toolkit-model.jar
2. Also to folder /lib, copy files:
  • ehcache-2.8.3.jar 
  • jackson-annotations-2.6.4.jar 
  • jackson-core-2.6.4.jar 
  • jackson-databind-2.6.4.jar 
  • logback-classic.jar 
  • logback-core.jar 
  • slfj-api.jar 
3. Edit or create if it doesn't exist, file toolkit.properties and place it in folder /WEB-INF/classes. The following properties are relevant:
  • cacheEnabled - boolean whether to use cache or not;
  • cacheMaxEntriesLocalHeap - integer the size of the cache;
  • cacheMonitorSeconds - integer the interval in seconds to perform cache stale checks;
  • cacheTimeToLiveSeconds - integer the number of seconds to keep an item in cache;
  • cacheTimeToIdleSeconds - integer the number of seconds to keep an idle item in cache;
  • contentRoot - string the path where the published Tridion pages are published (this path is defined in cd_storage_conf.xml);
  • jsonRoot - string the path where the JSON files are stored under;
In case dynamic tags are used, they can be configured using properties tag.name.X and tag.class.X. These tags define the tag name and tag fully qualified class name to be executed when the given tag name is encountered in the Component Presentation content. The X stands for a 1-based counter that allows us to define several tags by simply incrementing the counter.

Example:
The following definition of tags defines 2 tags. First with name mytag backed by class com.mitza.toolkit.tag.MyTag and the second with name yourtag and backed by class com.mitza.toolkit.tag.SecondTag:

tag.name.1=mytag
tag.class.1=com.mitza.toolkit.tag.MyTag
tag.name.2=yourtag
tag.class.2=com.mitza.toolkit.tag.SecondTag

At this moment, the Toolkit API can be executed in the web-application.

Logging

The Toolkit uses logback API. This can be configured in a logback.xml file in the web-application class-path (typically in WEB-INF/classes folder).

Add a logger on classes under com.mitza package and the logging will be output.

Sample logback.xml that outputs to console:

<?xml version="1.0" encoding="UTF-8"?>
<configurationscan="true">
<propertyname="log.pattern"value="%date [%thread] %5p %c{1} - %m%n"/>
<propertyname="log.level"value="info"/>

<appendername="STDOUT"class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>${log.pattern}</pattern>
</encoder>
</appender>

<loggername="com.mitza"level="${log.level}"/>

<rootlevel="OFF">
<appender-refref="STDOUT"/>
</root>
</configuration>



Toolkit - Examples

$
0
0
This post if part of a series about the File System Toolkit - a custom content delivery API for SDL Tridion.

This post describes a few examples of Toolkit API.

Model Factory

The following example retrieves a Page model and a Component model.

ModelFactory modelFactory = ModelFactory.INSTANCE;

TcmUri tcmUri =new TcmUri(6,1225,64);
PageMeta model = modelFactory.getModel(tcmUri);

tcmUri =new TcmUri(6,52);
ComponentMeta componentMeta = modelFactory.getModel(tcmUri);

Link Resolving

The following examples resolve Binary, Component and Page links.

LinkFactory linkFactory = LinkFactory.INSTANCE;

Link link = linkFactory.getBinaryLink(6,84);
if(link.isResolved()){
String url = link.getUrl();
}

// link to a Component
link = linkFactory.getComponentLink(6,52);
// link to a Component from a Page
link = linkFactory.getComponentLink(6,1138,1118);
// link to a Page
link = linkFactory.getPageLink(6,55);

Component Presentation Factory

The following example retrieves a Component Presentation by ids but also using highest linking priority.

ComponentPresentationFactory factory = ComponentPresentationFactory.INSTANCE;

ComponentPresentationMeta dcpMeta =
factory.getComponentPresentationWithHighestPriority(6,1151);
if(dcpMeta !=null){
// unresolved content
String content = dcpMeta.getContent();
}

dcpMeta = factory.getComponentPresentation(6,52,1117);

Component Presentation Assembler

The following example shows a returns the resolved Component Presentation content. If there are any tags in the content, they are resolved.

ComponentPresentationAssembler assembler = ComponentPresentationAssembler.INSTANCE;

String content = assembler.getContent(6,1151,1117);

Tag Handling

The following snippet executes tags found in the content. It replaces them with the actual output from the tags.

TagFactory tagFactory = TagFactory.INSTANCE;

String content = tagFactory.executeTags("some <mytag>quick brown fox</mytag> content");

The example above must backed by (1) the following tags definition in toolkit.properties file:

tag.name.1=mytag
tag.class.1=some.tag.example.MyTag

and (2) the tag class implementation, for example:

package some.tag.example;

importcom.mitza.toolkit.dynamic.TagRenderer;

publicclassMyTagimplements TagRenderer {

public String doTag(String tagBody){
// do actual processing here

return tagBody ==null?null: tagBody.toUpperCase();
}
}

The result content of the code above will be:

content ="some QUICK BROWN FOX content"

Dynamic Query

Example 1:
Simple query on Publication Id, Custom Meta that retrieves ComponentMeta models:

Criterion criterion =new AndCriteria(
newCustomMetaCriterion("Type","Article"),
newPublicationCriterion(13)
);

Query query =new Query(criterion);
List<ComponentMeta> componentMetas = query.executeComponentQuery();

Example 2:
Complex criteria using date range, Publication Id, Custom Meta that retrieves Page meta models. The following query retrieves PageMeta models from either publication 6 or 8 that have CustomMeta 'ReleaseDate' in a certain period of time AND that have CustomMeta 'Type' either 'Article' or 'News':

Criterion criterion =new AndCriteria(
Arrays.asList(
newOrCriteria(
newPublicationCriterion(6),
newPublicationCriterion(8)
),
newCustomMetaCriterion("ReleaseDate",new Date(1450673126000L),new Date(1550673206000L),false),
newOrCriteria(
newCustomMetaCriterion("Type","Article"),
newCustomMetaCriterion("Type","News")
)
)
);
Query query =new Query(criterion);
List<PageMeta> pageMetas = query.executePageQuery();

Example 3:
Pagination and Sorting example:

Criterion criterion =new CustomMetaCriterion("String","LOLEK2");
Query query =new Query(criterion);
query.addSort("Type", SortDirection.ASCENDING);
query.addSort(SortColumn.LAST_PUBLISH, SortDirection.DESCENDING);
query.setPage(2);
query.setPageSize(25);

List<String> uris = query.executeQuery();




Toolkit - JSON Models

$
0
0
This post if part of a series about the File System Toolkit - a custom content delivery API for SDL Tridion.

This post explains the format of JSON model files, which look typically like this:

{
"cp" : [ {
"c" : 10,
"cnt" : "Rendered CP content",
"pr" : 300,
"p" : 6,
"t" : 15
}, {
"c" : 10,
"cnt" : "Rendered CP content",
"pr" : 200,
"p" : 6,
"t" : 25
} ],
"cstm" : {
"meta" : {
"Float" : {
"k" : "Price",
"n" : [ 2.0 ],
"t" : "NUMERIC"
},
"Date" : {
"k" : "ArticleDate",
"d" : [ 72602000 ],
"t" : "DATE"
},
"String" : {
"k" : "Authors",
"s" : [ "Lolek", "Bolek" ],
"t" : "STRING"
}
}
},
"m" : 1450673231000,
"p" : 1482016384618,
"lnk" : [ {
"c" : 10,
"pg" : 20,
"pr" : 300,
"p" : 6,
"u" : "/page1.html"
}, {
"c" : 10,
"pg" : 30,
"pr" : 300,
"p" : 6,
"u" : "/page2.html"
} ],
"sch" : 46,
"tcm" : {
"p" : 6,
"i" : 10,
"t" : 16
},
"t" : "News Article Sample",
"mm" : false
}

The JSON above is saved in file 6-10.json, where 6 stands for the Publication id and 10 is the Component item id.

The model contains several parts: Component Presentations (cp), CustomMeta (cstm) and Links (lnk). Also it contains the last modified date (date when it was last saved in Tridion) - (m), last published date (p), Schema Id (sch), TcmUri (tcm), Title (t), Multimedia (mm).

The Component Presentation section contains all Dynamic Component Presentations on this Component and specifies their Component id (c), content (cnt), linking priority (pr), Publication Id (p) and Component Template Id (t).

The CustomMeta section contains all custom meta field and defines it as an array of metadata items (meta). Each item defining a key (k), a value and a type (t). The value can be one of three possible values: numeric (n), date (d), or string (s).

The Link section contains information about all links to this Component, such as Component Id (c), Page item Id (pg), link priority (pr), Publication Id (p), and page URL (u).

The TcmUri is represented as an object having a Publication Id (p), and Item Id (i) and a Type Id (t).

The static Component Presentations of a Component appear on the Page model that contains them. For example on the Page model below, there is one static Component Presentation specified by its Component Id (c), Linking priority (pr), Publication Id (p) and Component Template item Id (t).

{
"cp" : [ {
"c" : 10,
"pr" : 200,
"p" : 6,
"t" : 15
} ],
"f" : "/page3.html",
"m" : 1447029194000,
"p" : 1482016423976,
"pt" : 54,
"tcm" : {
"p" : 6,
"i" : 12,
"t" : 64
},
"t" : "Sample Page 3",
"u" : "/page3.html"
}

JSON models are placed in a folder structure under the jsonRoot folder specified in the toolkit.properties file.

The location of a JSON model file on the file system, under the jsonRoot folder is given by a path derived from the TCM URI of the item. The hash-code of the TcmUri is calculated, using Java's String.hashCode() method. Then we retain the last 4 digits of this hash-code. This gives us the 4-level deep path for the specified model.

Example:
We want to calculate the path for model with TCM URI: tcm:6-1225-64
1. First we calculate the string hash-code: 866716204
2. We retain the last 4 digits: 6204
3. We calculate the path: 6/2/0/4/
4. We append the TCM URI without prefix "tcm:", which gives: 6/2/0/4/6-1225-64.json
5. We prepend the jsonRoot folder: /my/jsonRoot/6/2/0/4/6-1225-64.json



Toolkit - Model Factory

$
0
0
This post if part of a series about the File System Toolkit - a custom content delivery API for SDL Tridion.

This post presents the Model Factory, an layer that offers CRUD operations on models backed by a cache and a file system provider.

The Model Factory is a singleton that allows the creation, retrieval, update and deletion of a model. It works using a CacheFactory and a File System Provider that acts as a data abstraction layer.

The Model Factory works on two types of models only: ComponentMeta and PageMeta. They are both generics passed to the factory methods or inferred from the returned type.

Method getOrCreate(TcmUri)

This method tries first to read a model from the cache. If it doesn't exist in cache, it tries to read it from the file system provider. If there is no such model on disk, then it will proceed to create a stub model that only has its TcmUri property set. The idea is that the user will populate the other model properties and will execute a updateModel.

public<T extends ItemMeta> T getOrCreateModel(TcmUri tcmUri){
T model;
String key = cacheFactory.getKey(tcmUri);
CacheElement<T> cacheElement = cacheFactory.get(key);

if(cacheElement ==null){
model = modelProvider.read(tcmUri);
}else{
model = cacheElement.getPayload();
}

if(model ==null){
model = modelProvider.create(tcmUri);
cacheFactory.put(key, model);
}

return model;
}

Method getModel(TcmUri)

This method is the most used method of the factory. It provides the read operation of a model from either the internal cache or from the file system provider. Usually the 'write' operations of this factory are called only by the Deployer extensions, during the publish/unpublish activities. The 'read' operation is used mainly in the Toolkit CD API.

public<T extends ItemMeta> T getModel(TcmUri tcmUri){
T model;
String key = cacheFactory.getKey(tcmUri);
CacheElement<T> cacheElement = cacheFactory.get(key);

if(cacheElement ==null){
model = modelProvider.read(tcmUri);
if(model !=null&& model.getLastPublished()==null){
model =null;
}
cacheFactory.put(key, model);
}else{
model = cacheElement.getPayload();
}

return model;
}

Method updateModel(T model)

This method is a write operation that persists the model sent as parameter to the file system. It also places the updated model in the cache.

public<T extends ItemMeta>void updateModel(T model){
TcmUri tcmUri = model.getTcmUri();
String key = cacheFactory.getKey(tcmUri);
model = modelProvider.update(model);
cacheFactory.put(key, model);
}

Method removeModel(TcmUri)

This method is a write operation that deletes a model from the file system and removes it from the cache.

publicbooleanremoveModel(TcmUri tcmUri){
String key = cacheFactory.getKey(tcmUri);
cacheFactory.remove(key);

return modelProvider.delete(tcmUri);
}



Toolkit - File System Provider

$
0
0
This post if part of a series about the File System Toolkit - a custom content delivery API for SDL Tridion.

This post explains the logic in the FileSystemProvider class that perform the actual CRUD operations on the JSON model files as it interacts with the underlying file system.

The provider handles also the serialization/deserialization between model objects and JSON by making use of the FasterXML Jackson serializer.

The FileSystemProvider is a singleton that implements the following methods:

Method create(TcmUri)

This is one of the trickier methods of the provider because it has to create different stub objects based on the type of the TcmUri passed as parameter. Namely, it will create either a ComponentMetaImpl or a PageMetaImpl object and it will populate its TcmUri property. I chose to implement this method using reflection, because it can provide more flexibility in the future, in case other types are required to be initialized.

public<T extends ItemMeta> T create(TcmUri tcmUri){
T result;
Class<T> aClass = ModelUtils.getClass(tcmUri);
try{
result = aClass.newInstance();
Method setTcmUri = aClass.getMethod("setTcmUri", TcmUri.class);
setTcmUri.invoke(result, tcmUri);
}catch(InstantiationException | IllegalAccessException |
NoSuchMethodException | InvocationTargetException e){
thrownewModelException(e);
}

return result;
}

Method read(TcmUri)

The method uses the PathMapper to identify the path of a JSON file on the file system, it retrieves it and then it performs a deserialize using the Jackson deserializer. It also identifies the type of the model to be returned by looking at the type inside the TcmUri passed as parameter.

public<T extends ItemMeta> T read(TcmUri tcmUri){
String path = pathMapper.getModelAbsolutePath(tcmUri);
File file =new File(path);
if(!file.exists()){
returnnull;
}

try{
Class<T> aClass = ModelUtils.getClass(tcmUri);
T model = serializer.deserialize(file, aClass);
return model;
}catch(SerializationException se){
thrownewModelException(se);
}
}

Method update(T model)

The method uses the PathMapper to identify the path of a JSON file on the file system, and if it doesn't exist, it creates the folder structure up until the model JSON file. It then uses the Jackson serializer to serialize the model passed as parameter and write it to the file system.

public<T extends ItemMeta> T update(T model){
TcmUri tcmUri = model.getTcmUri();
String path = pathMapper.getModelAbsolutePath(tcmUri);
File file =new File(path);

if(!file.exists()){
File directory = file.getParentFile();
if(!directory.exists()){
if(!directory.mkdirs()){
log.error("Directory {} was not created", directory);
}
}
}

serializer.serialize(file, model);

return model;
}

Method delete(TcmUri)

The method uses the PathMapper to identify the JSON model file on the file system and then it deletes it.

This is one of the tricker methods in the provider, because it needs to perform a cleanup once a directory is left empty after a model has been deleted.

publicbooleandelete(TcmUri tcmUri){
String path = pathMapper.getModelAbsolutePath(tcmUri);
File file =new File(path);

if(file.exists()){
if(file.delete()){
if(isCleanup){
deleteEmptyFolders(file.getParentFile());
}
returntrue;
}
}

returnfalse;
}



Toolkit - Cache Factory

$
0
0
This post if part of a series about the File System Toolkit - a custom content delivery API for SDL Tridion.

This post describes the Cache Factory the Toolkit uses to store commonly used models. The factory interfaces with an underlying EHCache instance named 'toolkit-cache'. Configuring the cache is possible either using a ehcache.properties file present at run-time in the application's class-path, or in the Toolkit configuration file toolkit.properties.

Configuring the Toolkit cache is described in post Installation and Configuration. Creating an EHCache programmatically is described in post Create an EHCache Programmatically.

The Cache Factory is used mainly by the Model Factory in its interaction with storing/retrieving JSON models.

The factory provides three main operations: get, put, remove.

The EHCache stores CacheElement generic objects that are simply Java beans around properties: payload of generic type and a last-check timestamp used in determining a stale value.

publicclassCacheElement<T>{

privatelong lastCheck;
private T payload;

publicCacheElement(T payload){
this.payload= payload;
lastCheck = System.currentTimeMillis();
}

// accessor methods below...
}

Method get(String)

This method looks up a value in the cache based on the given key passed as parameter. If the value is present in the cache, it is returned in the form of a generic CacheElement object with a specific payload type.

There is one additional stale check, which I explain in the next post Cache Invalidation.

public<T> CacheElement<T> get(String key){
Element element = cache.get(key);
if(element ==null){
returnnull;
}else{
if(isStale(element)){
remove(key);
returnnull;
}

CacheElement<T> result =(CacheElement<T>) element.getObjectValue();

return result;
}
}

Method put(String, T payload)

This method stores a given payload object in the EHCache under a certain key. If the key is already present, it simply replaces its value.

public<T>void put(String key, T payload){
Element element =new Element(key,new CacheElement<>(payload));

if(cache.isKeyInCache(key)){
cache.replace(element);
}else{
cache.put(element);
}
}

Method remove(String)

Finally, the remove method removes an entry from the EHCache corresponding to the given key.

publicbooleanremove(String key){
return cache.remove(key);
}



Toolkit - Cache Invalidation

$
0
0
This post if part of a series about the File System Toolkit - a custom content delivery API for SDL Tridion.

In previous post, I explained the use of a Cache Factory. This post describes a very simple cache invalidation mechanism the Toolkit uses in order to make sure it does not serve stale values (for a long time).

Given the nature of the Toolkit storage, i.e. files on a file-system, it is very easy to check when they were last published. By simply looking at the JSON model file last modified attribute, one can see the very moment that model was created/updated. This is the last publish time-stamp as well.

When a CacheElement is placed in cache, it is also given a last-check timestamp, which initially is set to now. This last-check will be greater than the JSON file last-modified attribute. The moment the JSON model is republished, the file last-modified file attribute will be greater than the last-check. When this happens, we know the CacheElement is stale and we remove it from cache.

The stale logic also checks for the existence of a JSON model file. An unpublished (a.k.a. missing) file will immediately be considered stale, and as such removed from cache.

In order to make the cache invalidation check more performant and to minimize I/O operations with the file system, we only check for stale elements on (a) get from cache operation and (b) when a certain interval has passed since the previous stale check. This interval is configurable in Toolkit configuration under property name cacheMonitorSeconds. This value represents a period of time that is acceptable to retrieve potentially stale objects. Typically this value should be low - i.e. 5, 10 or 30 seconds or as long as you're comfortable with. A value of 0 will perform the stale check on all calls to the cache.get() method.

More information about possible values in the toolkit.properties file is available in post Installation and Configuration.

privatebooleanisStale(Element element){
CacheElement<Object> cacheElement =(CacheElement<Object>) element.getObjectValue();
long lastCheck = cacheElement.getLastCheck();
long now = System.currentTimeMillis();

if(now - lastCheck > cacheMonitorInterval){
cacheElement.setLastCheck(now);
Object value = cacheElement.getPayload();

if(value instanceof IdentifiableObject){
IdentifiableObject identifiableObject =(IdentifiableObject) value;
TcmUri tcmUri = identifiableObject.getTcmUri();
PathMapper pathMapper =new PathMapper();
File file =new File(pathMapper.getModelAbsolutePath(tcmUri));

return!file.exists()|| file.lastModified()> lastCheck;
}
}

returnfalse;
}



Toolkit - Dynamic Linking

$
0
0
This post if part of a series about the File System Toolkit - a custom content delivery API for SDL Tridion.

In this post I describe the dynamic link resolving logic as part of the Link Factory.

There are three types of links: Component, Page and Binary links. Each of these links can be resolved using the Link Factory.

Component Link

Resolving a Component link implies finding the URL of the Page the target Component appears on. Using the current Toolkit models, it is quite straight forward to retrieve the URL, because the link information is contained within the Component model.

However, there might be several potential links available when performing Component link resolving. Namely, there can be several cases possible:
  • there is no linking information available in the Component model -- this means the link cannot be resolved (i.e. there is no Page published that contains the given Component);
  • there is exactly one Page available that contains the Component -- this means we retrieve the Page URL and return it as the link;
  • there are several potential Pages available that contain the Component -- in this case we need to pick one page only according to the following algorithm: 
    • take the Page that contains the Component with the highest link priority. If there are more than one pages possible, then go to next step;
    • take the Page relatively closest to the Page where the link is displayed on. The relative distance is the number of folders one page is away from the other. If there are more than one pages possible, then go to next step;
    • take the page that was published the latest;
You might notice that in order for the algorithm to work, we must provide the current Page -- this is the page where the link is displayed on. Without this page, we cannot identify accurately the relatively closest potential page.

public Link getComponentLink(TcmUri pageUri, TcmUri componentUri){
LinkImpl result =new LinkImpl();
ComponentMeta componentMeta = modelFactory.getModel(componentUri);
List<LinkInfo> linkInfos = componentMeta.getLinkInfos();
if(linkInfos ==null|| linkInfos.size()==0){
return result;
}

String[] urlParts;
PageMeta pageMeta = modelFactory.getModel(pageUri);
if(pageMeta ==null){
urlParts =new String[0];
}else{
urlParts = pageMeta.getUrl().split("/");
}

int pageId = pageUri.getItemId();
int maxPriority =1;
int minDistance = Integer.MAX_VALUE;
List<LinkInfo> filteredInfos =new ArrayList<>();

for(LinkInfo linkInfo : linkInfos){
if(linkInfo.getPage()!= pageId){
int distance = getDistance(urlParts, linkInfo.getUrl());
int priority = linkInfo.getPriority();
if(priority > maxPriority){
maxPriority = priority;
minDistance = distance;
filteredInfos.clear();
}elseif(distance < minDistance){
minDistance = distance;
filteredInfos.clear();
}
if(priority == maxPriority && distance == minDistance){
filteredInfos.add(linkInfo);
}
}
}

LinkInfo linkInfo = getLastPublished(filteredInfos);
if(linkInfo !=null){
result.setResolved(true);
result.setUrl(linkInfo.getUrl());
result.setTargetUri(new TcmUri(componentUri.getPublicationId(),
linkInfo.getPage(), ItemTypes.PAGE));
}

return result;
}

Below are the two helper methods getDistance between two paths and getLastPublished date out of a collection of Pages.

privateintgetDistance(String[] parts, String url){
int result =0;

String[] parts2 = url.split("/");
int n = Math.min(parts.length, parts2.length)-1;
int i =0;
boolean loop =true;

for(; i < n && loop; i++){
if(!parts[i].equals(parts2[i])){
loop =false;
i--;
}
}

result += parts.length- i -1;
result += parts2.length- i -1;

return result;
}

private LinkInfo getLastPublished(List<LinkInfo> linkInfos){
switch(linkInfos.size()){
case0:
returnnull;

case1:
return linkInfos.get(0);

default:
long maxPublished =0;
LinkInfo result = linkInfos.get(0);

for(LinkInfo linkInfo : linkInfos){
TcmUri metaUri =new TcmUri(linkInfo.getPublication(), linkInfo.getPage(), ItemTypes.PAGE);
PageMeta pageMeta = modelFactory.getModel(metaUri);
if(pageMeta !=null){
long lastPublished = pageMeta.getLastPublished().getTime();
if(lastPublished > maxPublished){
maxPublished = lastPublished;
result = linkInfo;
}
}
}

return result;
}
}

Page Links

Resolving a page link implies retrieving the Page model by TcmUri and returning its URL.

public Link getPageLink(TcmUri pageUri){
LinkImpl result =new LinkImpl();
PageMeta pageMeta = modelFactory.getModel(pageUri);
if(pageMeta ==null){
return result;
}

result.setResolved(true);
result.setUrl(pageMeta.getUrl());
result.setTargetUri(pageMeta.getTcmUri());

return result;
}

Binary Links

Resolving a binary link implies retrieving the Multimedia Component model and retrieving its link information URL. Binaries can be published using different variants, so we can either identify a link by its variant or, in the absence of a variant, simply serve the first link available.

public Link getBinaryLink(TcmUri binaryUri, String variant){
LinkImpl result =new LinkImpl();
ComponentMeta binaryMeta = modelFactory.getModel(binaryUri);
if(binaryMeta ==null){
return result;
}

if(!binaryMeta.isMultimedia()){
return result;
}

List<LinkInfo> linkInfos = binaryMeta.getLinkInfos();
if(linkInfos ==null|| linkInfos.size()==0){
return result;
}

variant = variant ==null?"": variant;

for(LinkInfo linkInfo : linkInfos){
String linkVariant = linkInfo.getVariant();
linkVariant = linkVariant ==null?"": linkVariant;
if(variant.equals(linkVariant)){
result.setResolved(true);
result.setUrl(linkInfo.getUrl());
result.setTargetUri(binaryUri);
break;
}
}

return result;
}



Tookit - Component Presentation Factory

$
0
0
This post if part of a series about the File System Toolkit - a custom content delivery API for SDL Tridion.

This post presents the Component Presentation Factory in the Toolkit. Its role is to retrieve a Dynamic ComponentPresentationMeta models based on several criteria.

The factory implements lookup methods by ids - Publication id, Component id, Component Template id and another one for getting a DCP based on the highest linking priority.

Method getComponentPresentation(int, int, int)

The three parameters, Publication id, Component id and Component Template id identify uniquely a Dynamic Component Presentation. If such a DCP is exists (i.e. is published), it id returned.

This method retrieves the Component model and then iterates over its DCP collection, returning the DCP with the given Component Template id, if available.

public ComponentPresentationMeta getComponentPresentation(int publicationId,int componentId,int templateId){
TcmUri componentUri =new TcmUri(publicationId, componentId);
ComponentMeta componentMeta = modelFactory.getModel(componentUri);
if(componentMeta ==null){
returnnull;
}

List<ComponentPresentationMeta> dcpMetas = componentMeta.getDynamicComponentPresentationMetas();
if(dcpMetas ==null){
returnnull;
}

for(ComponentPresentationMeta meta : dcpMetas){
if(meta.getPublicationId()== publicationId && meta.getComponentId()== componentId &&
meta.getTemplateId()== templateId){
return meta;
}
}

returnnull;
}

Method getComponentPresentationWithHighestPriority(int, int)

This method returns the first Dynamic Component Presentation that has the highest Component Template linking priority on a given Component. The parameters, Publication id and Component id are used to uniquely identify a Component model. Next, the DCP with highest priority is retained and returned.

public ComponentPresentationMeta getComponentPresentationWithHighestPriority(int publicationId,int componentId){
TcmUri componentUri =new TcmUri(publicationId, componentId);
ComponentMeta componentMeta = modelFactory.getModel(componentUri);
if(componentMeta ==null){
returnnull;
}

List<ComponentPresentationMeta> dcpMetas = componentMeta.getDynamicComponentPresentationMetas();
if(dcpMetas ==null){
returnnull;
}

int maxPriority =0;
ComponentPresentationMeta result =null;

for(ComponentPresentationMeta meta : dcpMetas){
if(meta.getPriority()> maxPriority){
maxPriority = meta.getPriority();
result = meta;
}
}

return result;
}




Toolkit - Component Presentation Assembler

$
0
0
This post if part of a series about the File System Toolkit - a custom content delivery API for SDL Tridion.

This post describes a Component Presentation Assembler factory. Its role is to retrieve a Dynamic Component Presentation and to trigger the execution of its content. Then the resolved content is returned.

The only method the assembler has is getContent(int, int, int), which takes parameters Publication id, Component id and Component Template id. It uses the ComponentPresentationFactory to retrieve the DCP.

Next, it triggers the execution of any custom tags might be in the DCP's content. The content can contain XML-like tags (e.g. <mytag>some body</mytag>). The tag support engine will execute all custom tags it finds in the content.

Finally, the resolved content is returned by the assembler and it is ready to be displayed on the response.

public String getContent(int publicationId,int componentId,int templateId){
ComponentPresentationMeta dcp = cpFactory.getComponentPresentation(
publicationId, componentId, templateId);
if(dcp ==null){
returnnull;
}else{
String content = dcp.getContent();
content = TagFactory.INSTANCE.executeTags(content);

return content;
}
}



Toolkit - Custom Tags Execution while Assembling

$
0
0
This post if part of a series about the File System Toolkit - a custom content delivery API for SDL Tridion.

This post presents the TagFactory in charge with execution of custom tags for the Toolkit Component Presentations.

The content of Dynamic Component Presentations can contain custom tags in the form of <mytag>some body</mytag>. These XML-like nodes are backed by a Java class that is executed when the tag is rendered.

The configuration of tag names and Java class name is done in file toolkit.properties. More information is available in post Installation and Configuration.

The custom tag support uses the following concepts:
  • TagFactory -- the entry point into execution of tags for a piece of content;
  • TagSupport -- class that associates a tag name with a tag implementation class;
  • TagRenderer -- interface that a tag implementation class must implement;

TagRenderer Interface

This is the interface that any tag class must implement. It contains only one method, doTag that is called by the TagSupport class when the tag is executed.

publicinterfaceTagRenderer{
String doTag(String tagBody);
}

A simple example tag follows:

publicclassMyTagimplements TagRenderer {
public String doTag(String tagBody){
return tagBody.toUpperCase();
}
}

TagSupport Class

This class represents an association between a tag name and a tag implementing class. The only operation this class defines is an execute(String) method. This calls method doTag(String) on the tag implementing class (the one that extends TagRenderer).

Each TagSupport contains a Regular Expression on the name of the tag, using simple pattern like <%1$s[^>]*>(.*)</%1$s>. The Regular Expression identifies the tag XML-like pattern and allows the extraction of the body for further processing.

public String execute(String tagBody){
try{
return clazz.newInstance().doTag(tagBody);
}catch(IllegalAccessException | InstantiationException e){
thrownewTagException(e);
}
}

TagFactory Class

The TagFactory is a singleton that contains an array of TagSupport objects. This array represents all the tags configured in file toolkit.properties.

Method executeTags(String) iterates over each tag and applies its pattern to the String content passed as parameter. For each match of the tag pattern in the content, it calls the tag's execute(String) method passing as parameter the identified tag body.

The original tag is then replaced with the result of its execution.

public String executeTags(String content){
StringBuffer result =null;
StringBuffer buffer =new StringBuffer(content);

for(TagSupport tag : tags){
result =new StringBuffer();
Matcher matcher = tag.getPattern().matcher(buffer);
while(matcher.find()){
String tagBody = matcher.groupCount()>0? matcher.group(1):null;
matcher.appendReplacement(result, tag.execute(tagBody));
}
matcher.appendTail(result);
buffer = result;
}

return result ==null? content : result.toString();
}



Toolkit - Dynamic Content Queries

$
0
0
This post if part of a series about the File System Toolkit - a custom content delivery API for SDL Tridion.

This post presents the Dynamic Content Query capability. The requirements for the Toolkit API are that it should be able to provide CustomMeta queries, pagination, and sorting -- all on the file system, without the use third party tools (database, search engines, indexers, etc). Therefore I had to implement a simple database engine and indexer -- which is described in more detail in post Writing My Own Database Engine.

The querying logic does not make use of cache. This means the query logic is executed every time. When models are requested, the models are however retrieved using the ModelFactory and those are cached.

Query Class

This is the main class for dynamic content queries. It is the entry point into the execution logic of a query. The class takes as parameter a Criterion (presented below) which triggers the execution of query in all sub-criteria of a Criterion object.

Class Query provides three main method: executeQuery, executeComponentQuery and executePageQuery, presented below. Each of these method return a list of items, paginated and sorted against the given parameters.

Method executeQuery

This method returns an list of Strings representing the TcmUris of the result. In terms of sorting, this is the most inefficient method because it first has to retrieve Component and Page models for all TcmUris, then sort them, then apply pagination on them and return only the result.

public List<String>executeQuery(){
List<String> uris =new ArrayList<>(criterion.executeQuery());

totalItemCount = uris.size();
uris = applyPagination(uris);

return uris;
}

Method executeComponentQuery

The method returns a list of ComponentMeta models that match the specified criteria. The method delegates the executeQuery call to the given Criterion and applies filter to item types Components only.

public List<ComponentMeta>executeComponentQuery(){
List<String> uris =new ArrayList<>(criterion.executeQuery(
newFilterImpl(ItemTypes.COMPONENT)));
List<ComponentMeta> result =new ArrayList<>(uris.size());
for(String uri : uris){
TcmUri tcmUri =new TcmUri("tcm:"+ uri);
ComponentMeta componentMeta = modelFactory.getModel(tcmUri);
if(componentMeta !=null){
result.add(componentMeta);
}
}

totalItemCount = result.size();
Collections.sort(result, sorter);
result = applyPagination(result);

return result;
}

Method executePageQuery

The method returns a list of PageMeta models that match the specified criteria.

public List<PageMeta>executePageQuery(){
List<String> uris =new ArrayList<>(criterion.executeQuery(
newFilterImpl(ItemTypes.PAGE)));
List<PageMeta> result =new ArrayList<>(uris.size());

for(String uri : uris){
TcmUri tcmUri =new TcmUri("tcm:"+ uri);
PageMeta pageMeta = modelFactory.getModel(tcmUri);
if(pageMeta !=null){
result.add(pageMeta);
}
}
totalItemCount = result.size();

if(sorter !=null){
Collections.sort(result, sorter);
}

result = applyPagination(result);
return result;
}



Toolkit - Dynamic Query Sorting and Pagination

$
0
0
This post if part of a series about the File System Toolkit - a custom content delivery API for SDL Tridion.

In the previous post Dynamic Content Queries, I presented a Query class that performs CustomMeta queries on JSON models published to the file system. This post presents the logic that sorts and paginates the result set.

The Query class defines four members that help the pagination and sorting logic:
  • Sorter -- the Comparator that perform the actual sorting of two ItemMeta objects (it returns an order between two ItemMeta instances);
  • page -- the page index to return out of all pages;
  • pageSize -- the number of items on a page;
  • totalItemCount -- the total number of items before pagination was applied;

Sorting

Sorting can be specified on different columns and can use different directions for each column (ascending, descending). A sort column is a predefined metadata of a model (title, last publish date) or a CustomMeta. An association between a sort column and sort direction is called a sort term. There can be several sort terms associated with a Query object.

Sorting is applied starting with each the first sort term. For items that are considered equal, the next sort term is applied to identify an order; the sorting logic continues until an order is established between two ItemMetas or when we run out of sort terms.

SortColumn

SortColumn class is an enum that defines what can we sort on:

publicenum SortColumn {
CUSTOM_META, LAST_MODIFIED, LAST_PUBLISH, TITLE
}

SortDirection

SortDirection class is an enum that defines the direction of sorting for a given column:

publicenum SortDirection {
ASCENDING, DESCENDING
}

SortTerm

This class is an association between a SortColumn enum, a SortDirection enum and potentially a String representing the CustomMeta name.

Sorter

This class implements the Comparator interface and it is a comparator of ItemMeta objects (i.e. ComponentMeta or PageMeta). The class holds a list of SortTerm object, which are applied in order when determining the order between two ItemMeta objects.

publicintcompare(ItemMeta i1, ItemMeta i2){
for(SortTerm sortTerm : sortTerms){
int compare = compare(i1, i2, sortTerm);
if(compare !=0){
return compare;
}
}

return0;
}

The specialized method compare(ItemMeta, ItemMeta, SortTerm) determines the order between the two ItemMeta objects according to the specified SortTerm:

privateintcompare(ItemMeta i1, ItemMeta i2, SortTerm sortTerm){
SortColumn column = sortTerm.getColumn();
SortDirection direction = sortTerm.getDirection();

switch(column){
caseCUSTOM_META:
String customMeta = sortTerm.getCustomMeta();
CustomMetaItem customMetaItem1 = getCustomMetaItem(i1, customMeta);
CustomMetaItem customMetaItem2 = getCustomMetaItem(i2, customMeta);
returncompare(customMetaItem1, customMetaItem2, direction);

caseLAST_MODIFIED:
Date date1 = getLastModified(i1);
Date date2 = getLastModified(i2);
return direction == SortDirection.ASCENDING?
date1.compareTo(date2): date2.compareTo(date1);

caseLAST_PUBLISH:
date1 = getLastPublished(i1);
date2 = getLastPublished(i2);
return direction == SortDirection.ASCENDING?
date1.compareTo(date2): date2.compareTo(date1);

caseTITLE:
String title1 = getTitle(i1);
String title2 = getTitle(i2);
return direction == SortDirection.ASCENDING?
title1.compareTo(title2): title2.compareTo(title1);

default:
log.error("Unknown sort column {}", column);
}

return0;
}

A couple of the helper method are presented below, such as getCustomMetaItem and compare(CustomMeta, CustomMeta, SortDirection). The goal in these method is to provide empty values instead of null such as the comparison can be performed correctly even for missing values.

private CustomMetaItem getCustomMetaItem(ItemMeta itemMeta, String key){
if(itemMeta ==null){
returnnull;
}

CustomMeta customMeta = itemMeta.getCustomMeta();
if(customMeta ==null){
returnnull;
}

CustomMetaItem customMetaItem = customMeta.getByName(key);
if(customMetaItem ==null){
returnnull;
}

return customMetaItem;
}

privateintcompare(CustomMetaItem meta1, CustomMetaItem meta2, SortDirection direction){
if(meta1 ==null&& meta2 ==null){
return0;
}elseif(meta1 ==null){
return direction == SortDirection.ASCENDING?-1:1;
}elseif(meta2 ==null){
return direction == SortDirection.ASCENDING?1:-1;
}

CustomMetaType type1 = meta1.getType();
CustomMetaType type2 = meta2.getType();
if(type1 != type2){
log.warn("Cannot compare custom meta {} with {}", type1, type2);
return1;
}

switch(type1){
caseDATE:
Date date1 = meta1.getDateValue();
date1 = date1 ==null? MIN_DATE : date1;
Date date2 = meta2.getDateValue();
date2 = date2 ==null? MIN_DATE : date2;
return direction == SortDirection.ASCENDING?
date1.compareTo(date2): date2.compareTo(date1);

caseNUMERIC:
float float1 = meta1.getNumericValue();
float float2 = meta2.getNumericValue();
if(float1 == float2){
return0;
}elseif(float1 < float2){
return direction == SortDirection.ASCENDING?-1:1;
}else{
return direction == SortDirection.ASCENDING?1:-1;
}

caseSTRING:
String string1 = join(meta1.getStringValues());
String string2 = join(meta2.getStringValues());
return direction == SortDirection.ASCENDING?
string1.compareTo(string2): string2.compareTo(string1);
}

return0;
}

Paginating

Once sorting is done, the results can be paginated. The routine below is a simple pagination logic that retains the indexes of a given 'page' out of the list of all items.

private<T> List<T> applyPagination(List<T> items){
if(page ==0&& pageSize ==0){
return items;
}

int offset = page <=1|| pageSize <=0?0: Math.min(items.size(),(page -1)* pageSize);
int limit = pageSize <=0?0: Math.min(pageSize, items.size()- offset);

List<T> result =new ArrayList<>(limit);
for(int i =0; i < limit; i++){
T item = items.get(offset + i);
result.add(item);
}

return result;
}



Toolkit - Criteria for Dynamic Queries

$
0
0
This post if part of a series about the File System Toolkit - a custom content delivery API for SDL Tridion.

In the previous post Dynamic Content Queries, I presented a Query class that performs CustomMeta queries on JSON models published to the file system. This post presents the logic that actually queries for results and applies different boolean operators for different Criteria.

The base class for each criteria is Criterion, which defines the base methods each criteria must implement:

publicabstractclassCriterion{
public Set<String>executeQuery(){
returnexecuteQuery(FilterImpl.EMPTY_FILTER);
}

publicabstract Set<String>executeQuery(Filter filter);
}

The Filter class contains a set of allowed Publication ids, the item type and whether to include the range boundaries or not. The Filter is used in modifying the results retrieved for each criteria from the indexes.

Each criteria uses an index to lookup values associated to a given key. For example, a CustomMetaCriterion uses a key and value to retrieve all ComponentMeta or PageMeta that have a CustomMeta on that key with the specified value.

In order to do this lookup in a performant manner, there is an index where all keys and values are available (in ascending order) and by simply looking up an key-value tuple, we can retrieve the associated TcmUris of all items that have such key-value as CustomMeta. More information about these indexes is available in post Writing My Own Database Engine.

The CustomMetaCriterion uses the DateIndex, NumericIndex or StringIndex to lookup custom meta keys, depending on the type of the custom meta value (date, numeric or string).

In its simplest form, for a Key-StringValue CustomMeta, the executeQuery method builds an index key on the CustomMeta key and StringValue, then performs the StringIndex lookup for such key.

The result is a set of Strings representing the TcmUris that correspond to that index key in the StringIndex.

public Set<String>executeQuery(Filter filter){
String indexKey = stringIndex.buildKey(key, valueString);
returnnew TreeSet<>(stringIndex.get(indexKey, filter));
}

The logical criteria (AndCriteria, OrCriteria) delegate their executeQuery to all sub-criteria and perform a logical intersection or union on the result sets. For example below, the AndCriteria below contains a list of sub-criteria that it needs to perform a logical AND between the result sets of each sub-criteria. As such, AndCriteria loops over the sub-criteria and then intersects their result sets:

public Set<String>executeQuery(Filter filter){
Set<String> result =null;
boolean first =true;

for(Criterion criterion : criteria){
Set<String> criterionResult = criterion.executeQuery(filter);

if(first){
first =false;
result =new TreeSet<>(criterionResult);
}else{
result.retainAll(criterionResult);
}
}

return result;
}



Viewing all 215 articles
Browse latest View live