# IT:AD:Design:Essays/The Repository Pattern # * [[../|(UP)]] {{indexmenu>.#2|nsort tsort}} * See: * https://medium.com/@jaimin_99136/generic-repository-with-ef-core-in-net-core-8-3e7a249b439a ## Prelude ## [EDIT] I’m publishing this post way too early. But I got a bit worked up with a colleague the other day who told me he ‘got’ repositories, as I watched him drag up references to EF right up to the UI layer…yuck. So…to gently – oh heck, not so gently, cause it was a bit annoying – suggest that there’s a lot more to things than the first [http://codeproject.com](http://codeproject.com) article he came across, here is a quick intro to some of my thoughts on the subject. The Repository pattern is important to get right, as it manages over a 3rd of the application, and if there are design flaws there, you will spend a lot of time and money chasing problems in the rest of the app that were caused from the repository. ## The Existing Status ## There are a lot of posts available on the net describing various ways of implementing a Repository pattern – most of them focusing on the concept of GenericRepository. Unfortunately, most……just don’t go far enough. Consider a current favourite GenericRepository contract. It goes something like this: #!c# public interface IRepository { int Count(); int Count(Expression> predicate); bool Contains(Expression> predicate); TAggregateRootEntity GetSingle(Expression> predicate); IQueryable GetAll(); IQueryable GetFiltered(Expression> predicate); void PersistOnCommit(TAggregateRootEntity aggregateRootEntity); void AddOnCommit(TAggregateRootEntity aggregateRootEntity); void AttachOnCommit(TAggregateRootEntity aggregateRootEntity); void UpdateOnCommit(TAggregateRootEntity aggregateRootEntity); void DeleteOnCommit(TAggregateRootEntity aggregateRootEntity); void DeleteOnCommit(Expression> predicate); //Internal, required by PersistOnCommit bool IsEntityNew(TAggregateRootEntity aggregateRootEntity); } There’s nothing especially wrong with the above. In fact, it’s got some good points: It doesn't make the mistake of having a `Commit()` method (it’s over in the `IUnitOfWork` class which I’ll cover later on) separating out from the What’s to be done (the Repository’s methods) from the When (the IUnitOfWork’s Commit method). But it’s also not…tight. It’s got some fuzzy thinking that can be cleaned up. ### Method Redundencies ### First of all, the more trivial issues that can be cleaned up are: There’s a useless `Count()` that is really the same as `Count(Expression> =null)` `GetAll()` should send off *big* alarms. Do you really want to allow some pipsqueek junior developer have access to a method that can return *all* elements in the database? Hum…maybe more thought is needed for that (see Repository As A Service further down). Second, `GetAll()` is nothing more than `GetFiltered`, except…without a `Filter`. So maybe it’s really a redundant method after all, that can be worked around with only one method definition with a default argument value. ###Specifications ### Removing the trivial garbage would improve things slightly, but not much. A more subtle improvement would be adding into the mix the use of the `Specification Pattern`. Notice the `Expression>` filter arguments on `Contains`, `GetByFilter`, `GetSIngle` etc. In a tightly controlled project, that can be factored out to a more appropriate `ISpecification`. The [[IT/AD/Patterns/Specification Pattern/]] is a rather elegant way of ensuring that you’re not letting a whole bunch of developers go add Linq Queries any old way – without some testing going on. By converting the Linq queries to a testable class, your risk is greatly diminished: #!c# /// /// A Contract for a Query Specification, /// that can be passed to a implementation of /// /// /// /// /// Implementation is as follows: /// /// { /// public Expression> IsSatisfied { /// get {return c => c.Country= "NZ";} /// } /// } /// ]]> /// /// /// /// Usage is then as follows: /// /// /// /// /// public interface ISpecification where TAggregateRootEntity : class { Expression> IsSatisfied { get; } } With that defined, the `IRepository` can be tightened up to only accept `Specifications`, ensuring that developers in the Domain layer are creating queries that are Unit testable. #!c# public interface IRepository { int Count(ISpecification specification); bool Contains(ISpecification specification); TAggregateRootEntity GetSingle(ISpecification specification); IQueryable GetFiltered(ISpecification specification); void PersistOnCommit(TAggregateRootEntity aggregateRootEntity); void AddOnCommit(TAggregateRootEntity aggregateRootEntity); void AttachOnCommit(TAggregateRootEntity aggregateRootEntity); void UpdateOnCommit(TAggregateRootEntity aggregateRootEntity); void DeleteOnCommit(TAggregateRootEntity aggregateRootEntity); void DeleteOnCommit(ISpecification specification); bool IsEntityNew(TAggregateRootEntity aggregateRootEntity); } Note that in the process I also got rid of `GetAll()` as mentioned, and the useless `Count()` method. ### Paging, Sorting, Includes ### But it’s still honestly not good enough. Turns out that what’s missing from the above is at least three more essential things: paging, sorting, and early-binding Includes. The first is fixable, by updating the `Repository` to include params that handle that. Note that for the remainder of the post, I’m going to go back to using the example before I converted the arguments to `Specifications`, as I think most users are not in that formal position, and it will make reading the post a bit easier. #!c# using System; using System.Linq; using System.Linq.Expressions; namespace XAct.Data.Repositories { public interface IRepository { int Count(Expression> predicate = null); bool Contains(Expression> predicate); //[Obsolete("Better to be explicit using other GetSingle")] //TAggregateRootEntity GetSingle(params object[] keys); TAggregateRootEntity GetSingle(Expression> predicate, object[] include = null); IQueryable GetByFilter( Expression> predicate=null, IPagedQuerySpecs pagedQuerySpecs = null, Func, IOrderedQueryable> orderBy = null, ); void PersistOnCommit(TAggregateRootEntity aggregateRootEntity); void AddOnCommit(TAggregateRootEntity aggregateRootEntity); void AttachOnCommit(TAggregateRootEntity aggregateRootEntity); void UpdateOnCommit(TAggregateRootEntity aggregateRootEntity); void DeleteOnCommit(TAggregateRootEntity aggregateRootEntity); void DeleteOnCommit(Expression> predicate); bool IsEntityNew(TAggregateRootEntity aggregateRootEntity); } } Everything is the same, except for the `GetByFilter` method. Notice that it has more arguments, that allow for defining sorting and paging in a vendor agnostic way. Let’s cover the `Paging` first. #### Paging Specifications #### One of the arguments specifies paging, and relies on the following interface: public interface IPagedQuerySpecs { int PageIndex { get; } int PageSize { get; } bool GetTotalRecords { get; } int PageRecordsReturned { get; set; } int TotalRecords { get; set; } void SetPageIndexAndSize(int pageIndex, int pageSize, bool getTotalRecordsFound=true); } fulfilled as follows: public class PagedQuerySpecs :IPagedQuerySpecs { private bool _initialized; public int PageIndex { get; private set; } public int PageSize { get; private set; } public bool GetTotalRecords { get; private set; } public int PageRecordsReturned { get { return _pageRecordsFound; } set { _pageRecordsFound = value; } } private int _pageRecordsFound = -1; public int TotalRecords { get { return _totalRecords; } set { _totalRecords = value; } } private int _totalRecords=-1; public void SetPageIndexAndSize(int pageIndex, int pageSize, bool getTotalRecords=true) { if (_initialized) { throw new ArgumentException("PagedQuerySpecs already initialized"); } GetTotalRecords = getTotalRecords; PageIndex = pageIndex; PageSize = pageSize; _initialized = true; } } All it’s doing is allowing for sending to the `GetByFilter` method a package of arguments required for paging. Nothing really genius here going on, but with it defined as an argument, the implementation can put it to good use when ready? The extra work of isolating the Paging specs into its own argument package allows one to later call the repository with clear paging instructions in a way that allows paging to be applied late (and that’s important…you’ll see why in a sec). What about the Linq statements Take and Skip? Yeah…what about them? They’re neat, but just cause they’re sweet, doesn’t mean you can use them any which way without considering their impact. Using Linq `Take` and `Skip` qualifications up in the Application layer (the layer invoking the Repository) ensures that your repository has to return IQueryable…and depending on your needs that might not always be a good idea. This ‘arguments as a package’ approach allows subsequent flexibility without having to rewrite swaths of code. Rewrites are costly. #### Sorting Specification #### The sorting makes use of a Linq statement that was originally intended for ORM providers to implement: IOrderedQueryable> orderBy There’s nothing else to at this point – just implement it, which I had better show… ## Implementing Filtering, Paging, and Sorting ## Right. We’ve covered Specifications (then put them aside for clarity’s sake), and covered defining paging/sorting in a generic way. How’s it implemented? public virtual IQueryable GetByFilter( Expression> predicate, IPagedQuerySpecs pagedQuerySpecs=null, Func, IOrderedQueryable> orderBy = null) { IQueryable resetSet = predicate != null ? this.ObjectSet.Where(predicate) : this.ObjectSet; if (pagedQuerySpecs!=null && pagedQuerySpecs.GetTotalRecords) { //Note that this will exec a second sql statement... pagedQuerySpecs.TotalRecords = (resetSet.Count()); } if (orderBy != null) { resetSet = orderBy(resetSet); } if (pagedQuerySpecs!=null) { int skipCount = pagedQuerySpecs.PageIndex * pagedQuerySpecs.PageSize; resetSet = skipCount == 0 ? resetSet.Take(pagedQuerySpecs.PageSize) : resetSet.Skip(skipCount).Take(pagedQuerySpecs.PageSize); } return resetSet; } ... var departmentsQuery = _exampleRepository .GetByFilter( paging: new PagedQuerySpecification(20,0), filter: contact => contact.Age >= 18 orderBy: q => q.OrderBy(d => d.Name)); So with the Sorting and Paging argument packages, the GetByFilter method has definitly improved its portability, exposting more of the underlying ORM’s flexibility in a vendor-agnostic way. ### Related Entities, Late Binding, et al…### There’s another snag in making a portable/generic Repository that we haven’t addressed yet.. Late Binding. Standard linq statements are fantastic – unfortunately not all of it is portable across all the various db linq providers. One of the glaring omissions is the Include statement. There is no portable way to say “Get me Customers AND their Addresses”. In EF, it’s done using a proprietary Include statement that takes the names of the Properties, in other ORMs it’s handled differently (taking types, etc.) So how can one make that portable? Easy. Once again, one can create a non-vendor specific Specification: public interface IIncludeSpecification { object[] Includes { get; } } which one could implement as follows: public class IncludeSpecification : IIncludeSpecification { public object[] Includes { get; set; } public IncludeSpecification(params object[] propertyNames) { Includes = propertyNames; } } Once that Specification is in place, once can use it to update our GetFilter contract as follows: public virtual IQueryable GetByFilter( Expression> predicate, IPagedQuerySpecification pagedQuerySpecs=null, Func, IOrderedQueryable> orderBy = null, IncludeSpecification includeSpecification = null) { IQueryable resetSet = predicate != null ? this.ObjectSet.Where(predicate) : this.ObjectSet; if (pagedQuerySpecs!=null && pagedQuerySpecs.GetTotalRecords) { //Note that this will cause a second sql statement to be executed... pagedQuerySpecs.TotalRecords = (resetSet.Count()); } if (includeSpecification != null && includeSpecification.Includes.Length>0) { foreach (var includeProperty in includeSpecification.Includes) { resetSet = resetSet.Include((string)includeProperty); } } if (orderBy != null) { resetSet = orderBy(resetSet); } if (pagedQuerySpecs!=null) { int skipCount = pagedQuerySpecs.PageIndex*pagedQuerySpecs.PageSize; resetSet = skipCount == 0 ? resetSet.Take(pagedQuerySpecs.PageSize) : resetSet.Skip(skipCount).Take(pagedQuerySpecs.PageSize); } return resetSet; } With the above parts in place, the GetByFilter method is exposing much of the ORMs flexibility in a non-vendor specific way. An example being something like: var departmentsQuery = _exampleRepository .GetByFilter( paging: new PagedQuerySpecification(20,0), filter: contact => contact.Age >= 18 orderBy: q => q.OrderBy(d => d.Name), include: new IncludeSpecification{"Addresses"}); Not only is that more elegant SOC, but it also is less costly to update in case you have to change ORM in the future (I know how I hear that one does not change ORM in mid course…but that’s bull. It does happen. It’s not fun, so as much as you have in place to allow for it not being the most costly operation ever, helps). In addition, you have a lot more than can be Unit tested, rather than integration tested. You’re code’s safer. You can sleep better at night. You can estimated better your exit costs. Note that this IncludeSpecification argument package can be added to the Get Single as well, to add to that method as well. Where are we? At this point, you have the following repository: namespace XAct.Data.Repositories { public abstract class EntityObjectContextRepositoryBaseBase : IRepository where TAggregateRootEntity : class { protected string EntityName; private ObjectContext ObjectContext { get { return _objectContext; } } private readonly ObjectContext _objectContext; private ObjectSet ObjectSet { get { return _objectSet; } } private readonly ObjectSet _objectSet; protected EntityObjectContextRepositoryBaseBase(IEntityObjectContext objectContext) { _objectContext = objectContext.InnerObject as ObjectContext; if (_objectContext == null) { throw new ArgumentNullException("objectContext"); } //Extract from the passed wrapper, the internal //EF specific class: //Note that the use //Covariance is not supported for something like: //IObjectSet _ObjectSet = // (IObjectSet)objectContext.CreateObjectSet(); _objectSet = objectContext.CreateObjectSet(); } public virtual int Count(Expression> filter=null) { return filter==null?this.ObjectSet.Count() : this.ObjectSet.Count(filter); } public bool Contains(Expression> filter) { //Can't do this as easily as one can with a CodeFirst Repo: //return this.ObjectSet.Contains(filter); return ((IRepository)this).GetByFilter(filter).Take(1).Count() > 0; } public TAggregateRootEntity GetSingle(Expression> filter, IIncludeSpecification includeSpecification = null) { //Get the ObjectSet out of the property: IQueryable resetSet = this.ObjectSet; //Add the includes if any //Note that each provider determines the format //in this case we are usign the string values of the properties //to include. Another ORM, it might be Types, etc. if (includeSpecification != null && includeSpecification.Includes.Length > 0) { // ReSharper disable LoopCanBeConvertedToQuery foreach (var includeProperty in includeSpecification.Includes) // ReSharper restore LoopCanBeConvertedToQuery { resetSet = resetSet.Include((string)includeProperty); } } return resetSet.FirstOrDefault(filter); } public virtual IQueryable GetByFilter( Expression> filter, IPagedQuerySpecification pagedQuerySpecifications=null, Func, IOrderedQueryable> orderBy = null, IncludeSpecification includeSpecification = null) { IQueryable queryableEntities = filter != null ? this.ObjectSet.Where(filter) : this.ObjectSet; if (pagedQuerySpecifications!=null && pagedQuerySpecifications.GetTotalRecords) { //Note that this will cause a second sql statement to be executed... pagedQuerySpecifications.TotalRecords = (queryableEntities.Count()); } if (includeSpecification != null && includeSpecification.Includes.Length>0) { // ReSharper disable LoopCanBeConvertedToQuery foreach (var includeProperty in includeSpecification.Includes) // ReSharper restore LoopCanBeConvertedToQuery { queryableEntities = queryableEntities.Include((string)includeProperty); } } if (orderBy != null) { queryableEntities = orderBy(queryableEntities); } if (pagedQuerySpecifications!=null) { int skipCount = pagedQuerySpecifications.PageIndex*pagedQuerySpecifications.PageSize; queryableEntities = skipCount == 0 ? queryableEntities.Take(pagedQuerySpecifications.PageSize) : queryableEntities.Skip(skipCount).Take(pagedQuerySpecifications.PageSize); } return queryableEntities; } public virtual void PersistOnCommit(TAggregateRootEntity aggregateRootEntity) { if (IsEntityNew(aggregateRootEntity)) { AddOnCommit(aggregateRootEntity); } else { UpdateOnCommit(aggregateRootEntity); } } public void AddOnCommit(TAggregateRootEntity aggregateRootEntity) { this.ObjectSet.AddObject(aggregateRootEntity); } public void AttachOnCommit(TAggregateRootEntity aggregateRootEntity) { this.ObjectSet.Attach(aggregateRootEntity); } public virtual void UpdateOnCommit(TAggregateRootEntity aggregateRootEntity) { ObjectStateEntry objectStateEntry; if (!this.ObjectContext.ObjectStateManager.TryGetObjectStateEntry(aggregateRootEntity, out objectStateEntry)) { this.ObjectSet.Attach(aggregateRootEntity); objectStateEntry = this.ObjectContext.ObjectStateManager.GetObjectStateEntry(aggregateRootEntity); }else { if (((int)objectStateEntry.State).BitIsSet((int)EntityState.Detached)) { this.ObjectSet.Attach(aggregateRootEntity); } } //Is this really needed? if (((int)objectStateEntry.State).BitIsNotSet((int)EntityState.Modified)) { Debug.Assert(true); objectStateEntry.SetModified(); //Or this way: //this.ObjectContext.ObjectStateManager.ChangeObjectState(aggregateRootEntity, EntityState.Modified); } //See: http://bit.ly/vVCecC //this.ObjectContext.ApplyCurrentValues(this.EntityName, aggregateRootEntity); } public void DeleteOnCommit(TAggregateRootEntity aggregateRootEntity) { ObjectStateEntry objectStateEntry; if (!this.ObjectContext.ObjectStateManager.TryGetObjectStateEntry(aggregateRootEntity, out objectStateEntry)) { this.ObjectSet.Attach(aggregateRootEntity); objectStateEntry = this.ObjectContext.ObjectStateManager.GetObjectStateEntry(aggregateRootEntity); } //Now that we have an objectStateEntry...are we detached? if (((int)objectStateEntry.State).BitIsSet((int)EntityState.Detached)) { this.ObjectSet.Attach(aggregateRootEntity); } //Now that we are attached....delete it: this.ObjectSet.DeleteObject(aggregateRootEntity); } public virtual void DeleteOnCommit(Expression> filter) { IQueryable aggregateRootEntities = ((IRepository)this).GetByFilter(filter); foreach (TAggregateRootEntity entity in aggregateRootEntities) { this.ObjectSet.DeleteObject(entity); } //return ForceCommit(forceCommitNow); } public abstract bool IsEntityNew(TAggregateRootEntity aggregateRootEntity); private string GetEntityName() { ObjectSet objectSet = this.ObjectSet /*as ObjectSet*/; return "{0}.{1}".FormatStringInvariantCulture(objectSet.EntitySet.EntityContainer, objectSet.EntitySet.Name); } //private bool HasChangesPending() //{ // // get the entities that have been inserted or modified // return (this.ObjectContext.ObjectStateManager.GetObjectStateEntries(EntityState.Added | EntityState.Modified | EntityState.Detached).Count()>0); //} } } Taking the above to work with EF4.2 (ie DbSet’s etc.) is a 3 minute affair – basically a search replace, and you would have a perfectly good Repository base class. Not so fast, buster… ## The UnitOfWork ## The above Repository, better than where we started off, is still a long way off from being robust. Looking back at the above base class, one can see that it expects to be injected, most probably via a ServiceLocator wrapping Unity, Nnject, or whatever floats your boat, a IUnitOfWork, that wraps an Entity Framework ContextObject. And that, my dear ladies is the rub…it’s only the most trivial apps that have only one context. Most complex apps have several. In order to work with a RepositoryService working against different ObjectContexts , you have several options. You could specialize the Repositories as needed. By that, I mean that there are times when you have a second db with the same schema, but different ConnectionString (this is a common strategy when dealing with tombstone deletion of records offline). As it is the SI that is factoring up the UoW to inject into the constructor of the Repository, it needs some hinting (by type) to know which UoW to factory up. That can be addressed by specialization of the repositories: public class TombstoneExampleRepo : ExampleRepo { public TombstoneExampleRepo (ITombstoneEFObjectContext objectContext): base(objectContext){} } public class ExampleRepo: EntityObjectContextRepositoryBaseBase { ... } //relying on: public class TomestoneEFObjectContext: EFObjectContext {...} That has the benefit that it works….but not for free… First of all, you have twice as many classes…although that’s not necessarily a bad thing when you consider that it’s just SOC. Secondly, it works. But as the SI is always returning the same Repo, and therefore same UoW (by type), one has only solved half the problem. One can now address several different databases, but one cannot have several UoW to the same Db. What’s missing is something smarter, something more specialized to the task of choosing the right UoW than just the ServiceLocator. ### UoWManager ### In fact, what we need is a UoWManager that can register several UoW, in a Stack, returning the Top one as the Current one. This changes the constructor …. oh hell….out of time.